What are the factors that will shape Peer Review in E-journals
Ross MacDonald
Introduction
This study examines the factors that will influence the future development of peer review in scholarly electronic journals (e-journals). The first scholarly journal, Philosophical Transactions, appeared in 1665 to facilitate the communication of findings among Royal Society Fellows, and the process of independent review arose from a desire to maintain quality standards (Meadows,1998, pp. 5-6). Today, this typically involves editors of scholarly(particularly scientific) journals obtaining independent advice on the merits or weaknesses of journal submissions. Ideally, such consultation provides a fair means of distinguishing between good and bad research, and alerts authors to necessary improvements in their work (van Rooyen, 2001). However, the new possibilities for information exchange in the online world raise questions about whether peer review is necessary at all, whether problems in traditional peer review can be addressed, what new forms of review will appear, and the place of quality control in new publishing models.
Will there be peer review at all?
Odlyzko (2002) has argued that the need for traditional peer review is overrated, and that in the online environment, the cumulative effect of hyperlinks or citations act as a form of peer review. Users of the Google[1]search engine may find this idea familiar. Odlyzko has further suggested that accessibility may be of greater significance in the online world than quality,so readers will settle for inferior (i.e. unrefereed) forms of papers that can be accessed more easily and cheaply – preferably for free (Odlyzko, 2002).
However, the underlying reason for the success of the journal in formal scholarly communication (particularly in science) is the high degree of credibility bestowed by peer review; so for the foreseeable future, some clearly defined form of peer review may be essential to any e-journal wishing to differentiate itself from the mass of self-published material on the Internet(Harnad, 1996; Halliday, 2001). A well-documented system of peer review where all participants are identified, and all reviews are available for viewing,would be a highly effective way of establishing a level of credibility similar to that of paper-based journals.
Addressing the problems of traditional peer review
Paper-based scholarly publishing can be a slow process, but the promise of more rapid publishing via the Internet is already having an impact on the traditional mechanics of peer review. Printed and electronic journals already conduct much of the review process using e-mail to send documents or communicate with reviewers and authors (Guernsey and Kiernan, 1999; Clark et al.,2000). An experiment in online peer review found referees were happy to assess manuscripts in PDF format rather than print (Wood and Hurst, 2000); the growing use of multimedia, and familiarity with working on-screen will likely see the entire mechanics of peer review conducted in the electronic environment. Internet-mediated telecommunications may allow inexpensive real-time interaction between editors, reviewers, and authors. There may also be a considerable degree of automation introduced: Sosteric et al. (2001) report their use of software to administer the review process for the e-journal The Craft[2]means that most procedures, from selecting referees to notifying authors of results, are automatic, while the editor is only required to make editorial decisions; they estimate a 50 percent reduction in editorial costs. The use of standard formatting languages, such as XML or HTML, may eliminate typesetting and printing, thus allowing immediate publication of electronic manuscripts(before or after review) with little or no human intervention.
Fully documenting the review process online will address some of the potential weaknesses of peer review itself. Although originality is a criterion for publication, truly original papers often receive negative reviews. Referees may be biased according to gender, nationality, or institutional affiliation(van Rooyen, 2001). They can fail to understand papers altogether, and miss errors or cases of fraud; some referees have themselves taken advantage of the peer review process to commit intellectual fraud (Wager and Jefferson, 2001). However, publishing the referees' reports linked to the actual paper would allow others to scrutinise the reports and any modifications made in the paper; for an example of this see the British Medical Journal online[3]. This approach is preferred by many publishers and scholars (Voelker, 2000), but would be prohibitively expensive in a paper-based publication.
Adapting peer review to different disciplines
Editors of e-journals may implement review procedures more responsive to the requirements of their target audience. Kircz (2001) notes that the importance of peer review varies between scholarly domains, and Weller (2001) has noted that different disciplines seem to be adopting a variety of online quality control models. Biomedical e-journals have adopted very traditional peer review processes, as ethical concerns in biomedical fields dictate strict review of findings before they can be applied to patients. In contrast, workers in the high-energy physics community have accepted the informal approach of the Los Alamos e-print arXive[4] where authors may or may not modify their papers in response to comments from readers. In the humanities, the scientific model of peer review is not always appropriate (Tomlins, 2001), and many scholars prefer a debate-style format; the on-going letters-to-the-editor commentary style used to complement traditional review in the interdisciplinary e-journal Psycoloquy[5]is a good example. The online journal Pediatrics[6] also uses this form of review (Anderson and Lucey, 1999), suggesting it may become more common in a variety of disciplines as a result of the enhanced interactivity possible on the Internet.
Peer review in different models of scholarly e-publishing
As e-publishing becomes more accepted by scholars, more organisations are becoming involved in scholarly publishing. Although scholarly journal publishing began with learned societies, the actual work of publishing has largely been taken over by commercial publishing houses because of the post-World War II boom in scientific publishing (Osburn, 1989). However, the dramatic growth in recent years in journal subscription costs has prompted initiatives by learned societies, libraries, and scholars to develop alternative publishing models that utilise the Internet as an inexpensive and accessible publishing medium.
Proposed scholarly publishing models incorporate varying roles for peer review. In paper-based journals, peer review is an inherent part of the publication process. Edmonds (2001) has noted that this has the effect of limiting what can be read to whatever is selected by the editors and review boards of scholarly journals. The appearance of various online public archives of scholarly papers, such as the Los Alamos e-print arXive, raises the possibility of placing all research in the public domain before any review process is undertaken, thus separating the processes of publication and quality control.
Ginsparg (2001)'s concept of a "global knowledge network" sees all research being self-archived by the author in online repositories such as the Los Alamos e-print arXive. These global archives would represent a basic "data"level of scholarly publishing; formal quality control would occur only later as part of higher level value-added services (e.g. e-journals) that would organise and synthesise access to papers in the archives based on subject, quality, or other criteria. This could involve the establishment of international electronic review boards that would assess unrefereed papers posted on personal or institutional Web pages by authors; the boards' assessments would then be collated to a set of publicly available online records that would include information necessary for finding the paper (Edmonds, 2001).
Nentwich (1999) has described several possible kinds of online scholarly archive, suitable for different purposes, and differentiated by the type of papers included plus the degree of quality control employed. This could range from strict (much like traditional peer review) to indirect or no quality control, as currently found in e-print archives. In contrast to this spectrum of possibilities, Harnad (1996) has concluded that quality control by peer review is the only area in which scholarly publishing on the Internet must resemble the traditional publication process. His model of scholarly e-publishing would involve the establishment of freely accessible, university-based archives where authors deposit their pre- and post-refereed papers (Harnad, 2001). Unlike Ginsparg (2001), Harnad distinguishes this process as "self-archiving"as opposed to "self-publishing", as papers would be peer reviewed in a quality control service supplied to the university by commercial publishers. Peer review is also integral to the Knowledge Exchange Model of scholarly e-publishing (Willinsky, 2001). This would see authors submitting papers to archives maintained by a global version of the Association of Research Libraries; peer review in a form appropriate to each discipline would be organised by professional academic associations, and the final paper would be made available in a "Public Knowledge Portal".
Along with new models for e-publishing, there may be new forms of scholarly paper. Kircz (2002) envisages the scholarly literature as becoming a kind of relational database, and describes a new form of paper composed of individual modules – self-contained portions of papers, which may or may not be re-used. Papers composed in such a way might actually increase the requirement for peer review; not only would the complete paper be subject to review, but each module would also be assessed on its first appearance in the literature.
Conclusions
Scholarly e-publishing is in a state of flux. Odlyzko (2002) has described e-publishing as a disruptive technology which has under-performed traditional scholarly communication, but has the potential to enable new applications and eventually outperform the traditional, paper-based model. The tradition of credibility through quality control that is inherent in present-day scholarly publishing will continue to play a role regardless of the exact publishing models used. However, the flexibility and interactivity allowed by the Internet will likely see a greater variety of approaches to peer review in the future.
Notes Google URL http://www.google.com/The Craft URL http://thecraft.icaap.org/British Medical Journal online URL http://bmj.com/Los Alamos e-print arXive URL http://arxive.org/Psycoloquy URL http://www.cogsci.soton.ac.uk/cgi/psyc/newpsyPediatrics URL http://www.pediatrics. org
ReferencesAnderson, K. and Lucey, J.F. (1999), "A new capability: postpublication peer review for pediatrics", Pediatrics, Vol. 104 No. 1, p. 106.Clark, A.,Singleton-Jackson, J. and Newsom, R. (2000), "Journal editing: managing the review process for timely publication of articles", Publishing Research Quarterly, Vol. 16 No. 3, pp. 62-71.Edmonds, B. (2001), "A proposal for the establishment of review boards", Journal of Electronic Publishing, Vol. 5 No. 4. Available at: http://www.press.umich.edu/jep/05-04/edmonds.htmlGinsparg, P. (2001), "Creating a global knowledge network", Second Joint ICSU Press – UNESCO Expert Conference on Electronic Publishing in Science, 19-23 February, UNESCO HQ, Paris. Available at: http://arxiv.org/blurb/pg01unesco.htmlGuernsey, L. and Kiernan, V. (1999), "Journals see the Internet as a tool in the peer-review system", The Chronicle of Higher Education,Vol. 45 No. 30, pp. A29-A30.Halliday, L. (2001), "Scholarly communication, scholarly publishing and the status of emerging formats", Information Research, Vol. 4 No. 6. Available at: http://InformationR.net/ir/paper111.htmlHarnad, S. (1996), "Implementing peer review on the net: scientific quality control in scholarly electronic journals", in Peek, R.P. and Newby,G. (Eds), Scholarly Publication: The Electronic Frontier, MIT Press,Cambridge, MA.Harnad, S. (2001), "For whom the gate tolls? How and why to free all refereed research from access- and impact-barriers online, now". Available at: http://www.cogsci.soton.ac.uk/~harnad/Tp/resolution.htmKircz, J.G. (2001), "New practices for electronic publishing 1: will the scientific paper keep its form?", Learned Publishing, Vol. 14 No. 4, pp. 265-72. Available at: http://www.catchword.com/alpsp/09531513 /v14n4/contp1-1.htmKircz, J.G. (2002), "New practices for electronic publishing 2: new forms of the scientific paper", Learned Publishing, Vol. 15 No. 1, pp. 27-32. Available at: http://www.catchword.com/alpsp/09531513/v15 n1/contp1-1.htmMeadows, A.J. (1998), Communicating Research,Academic Press, London.Nentwich, M. (1999), "Quality filters in electronic publishing", Journal of Electronic Publishing, Vol. 5 No. 1. Available at: http://www.press.umich./jep/05-01/nentwich.htmlOdlyzko, A.M. (2002), "The rapid evolution of scholarly communication", Learned Publishing, Vol. 15 No. 1, pp. 7-19. Available at: http://www.catch word.com/alpsp/09531513/v15n1/contp1-1. htmOsburn, C.B. (1989), "The structuring of the scholarly communication system", College & Research Libraries, Vol. 50 No. 3, pp. 277-86.Sosteric, M., Yuwei, S. and Wenker, O. (2001), "The upcoming revolution in the scholarly communication system", Journal of Electronic Publishing, Vol. 7 No. 2. Available at: http://www.press.umich.edu/jep/07-02/sosteric.htmlTomlins, C.L. (2001), "Just one more 'zine? Maintaining and improving the scholarly journal in the electronic present: a view from the humanities", Learned Publishing, Vol. 13 No. 1, pp. 33-40. Available at: http://www.catchword.com/alpsp/09531513/v14n1/contp1-1.htmvan Rooyen, S. (2001), "The evaluation of peer-review quality", Learned Publishing, Vol. 14 No. 2, pp. 85-91. Available at: http://www.catchword.com/alpsp/09531513/v14n2/contp1-1.htmVoelker, R. (2000), "Publishers debate future of online journals", Journal of the American Medical Association, Vol. 284 No. 8, pp. 943-4.Wager, E. and Jefferson, T. (2001), "The shortcomings of peer review in biomedical journals", Learned Publishing, Vol. 14 No. 4, pp. 257-263. Available at: http://www.catchword.com/alpsp/09531513/v14n4/contp1-1.htmWeller, A.C. (2001), "Peer review in an electronic environment", Editorial Peer Review: Its Strengths and Weaknesses, Information Today, Medford, NJ.Willinsky, J. (2001), "Proposing a knowledge exchange model for scholarly publishing", Current Issues in Education, Vol. 3 No. 6. Available at: http://cie.ed.asu.edu/volume3/number6/index.htmlWood, D. and Hurst, P. (2000), "Online peer review: perceptions in the biological sciences", Learned Publishing, Vol. 13 No. 2, pp. 95-100.
Ross MacDonald(merrimac54@hotmail.com) is an independent consultant based in Kuala Lumpur, Malaysia, and is also associated with the School of Information Management of Victoria University of Wellington, PO Box 600, Wellington, New Zealand.
