First Monday

JIME: An Interactive Journal for Interactive Media, by Simon Buckingham Shum and Tamara Sumner
How can new media positively transform scholarly practices? In this article, we describe the Journal of Interactive Media in Education (JIME). JIME's peer review process is designed to promote multidisciplinary dialogue through the use of a purpose-designed Web document-discussion interface. This innovative peer review model and the resulting enriched digital documents illustrate some of the possibilities for promoting knowledge construction and preserving intellectual products in digital scholarly publications. We present JIME's technical infrastructure, editorial policy, and peer review process, and discuss how these features are used to support the journal's goals. Finally, we conclude by considering what aspects of our approach might be suitable for e-journals in other disciplines.

Contents

Introduction
A Unique convergence of e-journal features
Discussion
Conclusion

++++++++++

Introduction

How can new media positively transform scholarly practices? One possible way is for scholarly publications such as electronic journals (e-journals) to better support the needs and practices of their particular users, instead of the publishing process. Traditional academic publishing contributed to scientific culture by creating "immutable mobiles" (Latour, 1987), that is, static representations in the form of published documents that traveled across space and time to disseminate new ideas. This publishing model is linear and one-way, separating research (the process) from results (the product and practices), and the producers of the research from the peer community. This model is largely anonymous, providing little opportunity for feedback from the larger community either before or after publication, and it assumes a stable environment where it is acceptable if communication delays between producers and consumers are measured in years (Marion and Hacking, 1998).

Given the current rapid pace of scientific innovation, simple dissemination is no longer an adequate model to ensure that science moves forward by building on the results of others. Rather, integrated approaches are needed that provide: (1) opportunities for community-wide collaboration, negotiation, and knowledge construction early in the publishing process, and (2) mechanisms to publish a wider array of intellectual products for community-wide sharing and reuse. For the past few years, we have been exploring such an approach to scholarly publishing in the context of an electronic journal.

JIME is a freely available e-journal, published since 1996, targeted at researchers and practitioners interested in educational technology. JIME was founded with three goals in mind. First, as with most journals, it is intended to be a forum for innovative work in its field. Second, rather than simply reading about interactive media, we wanted to make it possible for readers to directly experience the systems and techniques being described. Third, we believed that this multidisciplinary field could best be advanced by bringing together people reflecting the field's multiplicity of perspectives. Educational technology is located at the intersection of human sciences (including psychology, education, and anthropology) and computing sciences (including human-computer interaction, artificial intelligence, and system design). These fields have profoundly different theoretical underpinnings, research methods, modes of discourse, and criteria for publication that are challenging to bridge. Of particular concern, is the persistent gap between educational theory and educational practice; very few educators read or contribute to educational research journals (Pea, 1999). Thus, one of our major goals was to bridge these disciplinary and experiential gaps by fostering dialogue between participants from diverse backgrounds (e.g., researchers, educators, system designers, and policy makers) and distant geographic locations.

Towards this end, JIME's peer review process is designed to promote multidisciplinary dialogue through the use of a purpose-designed Web document-discussion interface, which tightly links the article to an area for review comments and discussion. This innovative peer review model and the resulting enriched digital documents illustrate some of the possibilities for promoting knowledge construction and preserving intellectual products in digital scholarly publications. In the remainder of this article, we present JIME is technical infrastructure, editorial policy, and peer review process, and discuss how these features are used to support the journal's goals. Finally, we conclude by considering what aspects of our approach might be suitable for e-journals in other disciplines.

++++++++++

A Unique convergence of e-journal features

Lancaster (1985) proposes a continuum for classifying journals according to the extent to which they use computer technology. At Stage 1, a journal uses computers simply to produce print, progressing to Stage 6 as follows:

  1. computers used for print production
  2. journal distributed in both print and electronic formats
  3. publication design is rooted in print, but articles are developed solely for electronic distribution
  4. interaction between authors and readers is possible; publications can evolve as a result of such interactions
  5. the inclusion of multimedia content
  6. both interactive participation and multimedia capabilities are supported

JIME's unique set of features qualifies it as a Stage 6 e-journal on Lancaster's continuum. More specifically, through technology, editorial policy, and review process design, JIME tries to use its multimedia capabilities and interactive participation to support individual and collective interpretation around a shared, central document. These goals are motivated by work pointing to the centrality of negotiation and collective interpretation around shared documents in the formation and maintenance of scientific communities (Anderson, 1983; Brown and Duguid, 1996; Fish, 1980; Kuhn, 1996). We are exploring how surrounding academic documents with related secondary resources (e.g., survey data, video clips of use, demonstrations, positive and negative commentary) might support scholarly interpretation, for instance, judging the originality of ideas, quality of results, and appropriateness of methods. By focusing on interpretation, we hope to provide a forum where people from different backgrounds can begin to develop a shared understanding of research concerns, research methods, important concepts and vocabularies, and key theories in the area of educational technology and interactive media in education.

Technology: Document-Centered Discourse Interface

At the technology level, JIME provides a document interface that enables users (readers, reviewers, authors) to progressively enrich JIME documents with important secondary resources, including interactive demonstrations, video and audio clips, evaluation instruments, discussions, and pointers to related or future work (Figure 1). The rationale and human-computer interface considerations that went into this document interface design are fully described in Sumner and Buckingham Shum (1998a; 1998b). The central document is shown in the left pane of Figure 1. Most of the review process takes place using the document interface, augmented by e-mail. The discourse (i.e., discussions between readers, authors, reviewers, and editors) is shown in the right pane. A key aspect of our design is the integration between the document and the discourse, where links to the discourse are embedded directly into the document form itself (e.g., the comment icons at the start of every section heading). Integrating document and discourse within the same environment assists smooth switching between reading and commenting; one can immediately see who has commented on a given theme or section, or quickly display the relevant part of the article to which another participant may be referring. We refer to this functionality as 'document-centered discourse'.

This document-centered discourse interface is very link-rich, making the publication of documents with associated discourse intensive in time and effort. To make the publication of these documents tractable, we created a publishing toolkit called D3E to automate large parts of the mark-up and publication process [1]. To date, we have used this toolkit to create document-centered discourse sites in numerous contexts, including two e-journals (Buckingham Shum and McKnight, 1997; JIME), a national policy debate (Dearing, 1997), and an academic conference making innovative use of digital and face-to-face modes of communication (Learning.Org). Our experiences across these sites indicate that the technology alone is insufficient to ensure that (1) discourse occurs and (2) that it serves the desired goal of supporting interpretation. By far the most important factor is the redesign of policy and processes.

 

Figure 1: JIME's document interface.

On the left is the Article Window, on the right the Commentaries Window showing the top level outline view of discussion about the document. Key: [1] Comment icon embedded in each section heading: clicking displays section-specific comments; [2] active contents list extracted from the section headings; [3] print versions as HTML and PDF; [4] numeric or author/date citation automatically linked to corresponding reference in footnote window; [5] a reverse hyperlink is inserted for each citation of a reference; [6] an editorial note to draw attention to a controversial issue in the author-reviewer debate that 'made it' into the published version; [7] section-specific review comment; [8] an editorial comment summarising the review discussion and specifying change requirements. (Note that there are two versions of the user interface: one as shown, and for smaller displays, the document and discussion are placed in separate browser windows.)

Policy: Encourage Interactive Materials

Much research into educational multimedia critiques and/or proposes examples of multimedia in learning contexts. In this capacity as a design discipline, the field needs to be able to communicate multimedia designs using more than text and screenshots of systems. Since the issue of interactivity lies at the heart of the research, the best way to communicate new work, and enable reviewers and readers to assess the claims being made, is to provide access to part or all of the design in question. JIME's policy therefore strongly encourages authors to provide access to their work where relevant, and we provide technical advice and assistance as far as possible in helping them to do so.

As illustrated in Figure 2, JIME articles may contain examples either embedded in the text using Web browser plug-ins [2], or downloadable to run on readers' machines [3]. Guided tours (using a screen recording) with optional commentary from the author are another way to communicate work more effectively than through static screenshots [4]. As much research in this field also involves the use of Web sites, it is simple for authors to provide direct links into a live site, or a specially prepared version (e.g. with a guided tour) for readers to see for themselves what they have been reading about [5].

In addition to design examples, an obvious area in which eJournals can add scientific value is in providing access to audio or video data. Extracts from transcripts notoriously hard for a reader to get into when separated from source can now be interpreted in the light of the original conversation or video clip embedded in the text [6].

Finally, JIME provides a forum for authors to experiment with new forms of scholarly communication, which depart from traditional, hierarchical texts. While the Web is a hypertext system, it is generally used by scholars to disseminate traditional documents. However, more radical forms of hypertextual essay are beginning to emerge, and JIME encourages its more adventurous authors to experiment [7].

 

Figure 2: Examples of interactive multimedia components in JIME articles.

[1] Interactive extract from an art history CD-ROM enabling readers to play with the construction of a painting; [2] interactive Java applet for visualizing code execution over the internet; [3] extract of video showing children programming a robot; [4] introductory walkthrough of an economics package with commentary from the author [See main text for links to these examples].

Process: Dialogic Peer Review

JIME has pioneered a dialogic peer review process, in which authors and reviewers are introduced to each other, and conduct a review debate (Figure 3). On receipt of a submission, an editor will judge whether the basic relevance and substantiveness of the article merits the investment of energy by two to six reviewers. When an article is judged to be relevant to the journal, the publisher (often the same person as the editor) uses the D3E toolkit to create a secret review site for that article resulting in the document-centered discourse interface shown in Figure 1. It is then published as a private preprint on the journal site at a URL known only to the editor, authors and reviewers.

The editor solicits reviewers and when all reviewers (typically three or four) are arranged, the editor uses e-mail to introduce the participants (authors and reviewers) to each other and brief them on the review process. A review schedule is agreed, including a period of at least two weeks during which all reviewers are able to respond in a timely manner to replies from the author to their reviews. Experience has shown that the most fruitful and stimulating discussions take place when authors reply to reviewers, and vice-versa.

Next, for a three or four week private review period, reviewers and authors discuss and debate the article. While reviewers may choose to remain anonymous, journal policy is to encourage named review and, with only a couple of exceptions, all reviewers to date have done so. During this period, editors support the debate process in many ways. For instance, we may need to answer questions participants have about the process or the technologies. Or, we may need to remind authors that they are not only allowed, but encouraged, to participate. Authors, reviewers and the acting editor are automatically subscribed to an article under review, receiving server-generated e-mail copies of postings to its discussion space.

 

Figure 3: The JIME review lifecycle.

This cycle includes private and public open peer review phases, with active stakeholders involved at different points.

When the private review deadline is reached, the editor formulates a summary of the key points made by reviewers which justifies an accept/reject decision. This is conventional journal procedure, except that this editorial is then threaded into the review discussion as one or more comments. If the submission is deemed acceptable (pending changes) the preprint is then linked to JIME's front page (with the clear status of Preprint). The preprint is then announced to relevant communities and readers in general, who are invited to view the review debate that has taken place and contribute their own insights. At this point, any readers interested in tracking an article can also subscribe to it, and receive e-mail updates when new comments are added. This public review phase lasts for a month (a little longer over holiday periods), after which any final editorial comments are sent to the authors, who then revise.

In addition to making the review process visible, this approach does appear to foster multidisciplinary debate. Figure 4 shows the outline of a thread with contributions from reviewers, the author, the editor, and readers. The participants were engaging in collaborative knowledge construction across their disciplinary boundaries as they discussed the meaning of strong versus weak multimedia. One reviewer offers a system-oriented definition based on immersion: virtual reality is strong multimedia and audio-graphics is weak. The author, an economist, offers a definition based on affordable accessibility; strong multimedia systems are those that are widely accessible by the intended user group. A reader with a background in educational theory suggests instead that strong multimedia are those that best serve the pedagogical aims. This insightful debate was published along with the final article, and the article was enriched to contain links to this part of the review debate.

 

Figure 4: Outline of a review debate on the originality and importance of ideas.

Editorial comments play several important roles in JIME, beyond the conventional meta-review summary. One important form of editorial comment concerns suggesting and promoting new forms of hypermedia literacies. For instance, as shown in Figure 5, we try to demonstrate and promote interlinking between: (1) the article and its associated review debate and (2) other articles.

 

Figure 5: Editorial modelling of threaded hypertext contributions to online peer review debates with cross-linking from one review discussion to another.

Sometimes, instead of requesting authors to modify part of their article in response to a comment, the editor will instead suggest to authors that they respond in the review debate and link from the text to this part of the debate. As an example, Figure 6 shows how authors linked from within their published article back to a particularly interesting thread in the article's review discussion. Such linking enables authors to use the review discussion as a form of amplifying footnote. In this way, the narrative flow of the central document is preserved, but the intellectual effort invested in the review process is re-used by drawing readers' attention to the availability of this secondary resource.

 

Figure 6: Review discussion.
The persistence of review discussion makes it a resource for authors to point readers to. This screenshot shows part of a published JIME article on the left, and the review discussion for that section on the right. The authors have inserted a link in their final text taking the reader to an interesting discussion thread that arose during the review process, preserved with the final publication, and available for readers to respond to.

It will be apparent that the role of Acting Editor on a submission has evolved in JIME. This now involves setting up and overseeing a review discussion. We have found that this role needs to be facilitated for new editors, by sending them editor-oriented summaries of the review process, and example template e-mail messages for them to customize to their own taste before introducing authors and reviewers to each other. It is fair to say that to date, the majority of reviews have been edited by the authors.

After the public review period, the authors modify the article in response to the review debate and the editorial meta-review. When the editor receives the final article and judges the modifications to be acceptable, the editor then edits the review debate to determine which parts will be published with the final article. Low-level comments pertaining to writing style or syntax are removed since these should have been addressed in the rewrite. Likewise, comments suggesting how to change parts of the article that have been addressed are also removed. The editor ensures that the context that the comment pertained to still exists. If it does not, the comment is removed. Sometimes the editor will ask reviewers if they wish to modify a specific comment or add another one in light of changes in the article. Often the comments left after this culling are those related to broader theoretical or methodological issues, related experiences or systems, ancillary questions, etc. These comments provide readers with an 'interpretive space' around the article, allowing them to see not only a polished (possibly multimedia) publication, but also some of its intellectual history: what the review participants thought of it.

The final version is then published (and announced to the JIME news list), with the edited version of the review debate remaining open for further comment. As shown in Figure 7, it is even possible to enrich documents with pointers to future work that takes place after publication.

 

Figure 7: Additions and changes.
In an article's review discussion, the possibility is raised of developing a Web version of the system described in the article. A year after publication, the editor posts an UPDATE pointing to a new project to do precisely this.

++++++++++

Discussion

The Merits of published, open peer review debates

It is not a goal of this article to review research into different peer review models. Suffice to note that there is a substantive literature which has established the limitations of closed (anonymous) peer review (e.g. Lock, 1985; WAME, 1994-2001), that there is no evidence that open peer review damages the quality of reviews, and some evidence that it may improve it in certain respects (e.g. Godlee and Jefferson, in press; Goldbeck-Wood, 1999; van Rooyen et al., 1999; 1998). This research, however, has been conducted in the context of traditional peer review processes in paper journals, and in other disciplines. JIME's online, interactive review process changes several variables all at once: debates that are open, public, during and following publication, in an online medium. We have not conducted controlled studies to compare JIME's process with a conventional one. However, we do note that in medicine, one of the more conservative domains given the importance of publishing reliable research, there is an active debate on the quality of peer review. Established journals, especially those with more advanced e-journal services, are now seriously considering open peer review as standard policy [8].

Beyond the purely functional arguments, we are also sympathetic to the ethical argument that open review addresses the central weakness of anonymous peer review, namely, that we do not accept secret decision-making on the quality of public work in any other domain, so why for scholarly work? Those who wield this power need to be accountable. It is hard to be more transparent about the review process than JIME, beyond perhaps publically archiving all versions of the submission prior to the final version, plus review comments, and all other correspondence with the author. At present, we maintain a private archive of all submissions and their debates for research purposes, on the basis that authors will want only the most polished version available, and that readers will not be interested in the low level review comments that are edited out of the final, published review debates.

Moreover, as we are also implementing a variation of what Harnad (1996) has called scholarly skywriting, it seems particularly appropriate to allow participants to know to whom they are talking, helping them interpret what others are saying. Scholarly skywriting, as implemented in Harnad's journal Psycoloquy [9], filters all contributions to a discussion via an editor, whereas in JIME we do not burden an editor with this responsibility. We have yet to encounter the need to edit or delete inappropriate review comments; reviewers and readers adopt the expected netiquette and sense of responsibility for what they say, and with the odd exception, the forums are too small and specialist to make it worthwhile for random outsiders to post controversial flames or other irrelevant material.

New medium, new messages

In JIME we see the emergence of some additional phenomena to those already described. Firstly, reviewers and authors often negotiate amongst themselves over how to deal with an objection raised in the review discussion. Typically, they will iterate round one or more cycles of clarification until the author agrees to a particular change. Thus, the reviewer's role has to some extent blurred into that of the editor in helping to move from critique to change requirements.

Secondly, whilst it has always been the case that authors, reviewers and editors may not agree, (a) this has been hidden to all but those involved, and (b) sometimes the resulting debate is worth preserving, as demonstrated by journal special issues in which commentators contest issues with each other and authors. JIME both legitimises and gives voice to dissenting views, alerting the reader that there may be a controversy behind a text, even though it has been accepted (see Figure 1 for instance). From a literary theory perspective, this paratext serves to undermine the impression of closure that scholarly texts often seek to convey rhetorically through their form and content. A scholarly article is merely a milestone, a contribution to an ongoing discourse.

Also, from an educational perspective, we have heard from a few JIME readers that faculty sometimes show JIME review debates to their students. Perhaps in this manner research students will benefit from gaining backstage access to the workings of the review process, as they learn to participate appropriately in the different genres of discourse within their research community.

Co-evolve the peer review process with the new medium

There are many e-journals offering 'Add Comment' buttons on published articles, but anecdotal reports suggest that these are used very rarely. JIME differs fundamentally in that it implements policies and a review process that are tailored to the characteristics of the new medium, rather than bolting on an interactive facility after the review and revision is completed. Busy readers have little motivation to post comments on published work that they cannot influence, a fact borne out by an informal control condition that makes for interesting comparison with JIME. The first author edited a special issue for another journal, which conducts review in the traditional way. The final papers were published on paper, but also on the Web using D3E, with linked discussion spaces for commentary [10]. However, there have been only a few posts to this site since 1997, despite the fact that it is pointed to from most of the top sites in the field; researchers are simply downloading the papers.

Our conclusion is that to use the Web for productive peer review discourse, it is critical to appoint reviewers (not just hope that someone out there will comment), and set dates and deadlines. Engaging reviewers in a social contract, and focusing energies for an agreed period, moves a review discussion from a dormant to active state, making the whole process dynamic. To improve the discussion dynamics, we have recently shortened our typical private review period from four weeks to three weeks with positive results.

Since we work hard to find reviewers who are experts in the field (sometimes researchers whose work has been cited in the submission), authors report gaining far more from engaging in critical discussion of their work than they get in conventional review. Reviewers undoubtedly give more to a review discussion, but are interested in the work, curious to experience JIME's review process, and know what will be involved before agreeing to serve as a reviewer. A few reviewers have even told us that reviewing articles for JIME is fun!

Would this work in your field?

A question that must be asked of any innovation in scientific publishing concerns the specificity of the factors that made it a success (see for instance discussion of the success of the Los Alamos eprint archive: Ginsparg, 1996; O'Connell, 2000). Factors will include funding models, technological infrastructure and literacy, and of particular relevance in this case, the characteristics of the discipline's research and modes of discourse.

In 1996, Harnad (1996) reflected that the Net was a medium about which most serious scholars are still quite wary. There are undoubtedly still scholars for whom the Internet is still irrelevant, or too confusing, but in 2000, most scholars recognise that they cannot afford to be without the internet. For many, life without it would be a major step backwards. More advanced technologies such as Web multimedia and Java-based applications remain the province of the early adopters, but basic e-mail and Web literacy is becoming ubiquitous.

JIME has to date been funded by the Open University and a range of research grants concerned with developing the D3E infrastructure required to publish hypertext documents and discussions (which has found many applications beyond e-journals). We are keen to maintain free access, since charging to participate in public open peer review debates would very likely dampen or even kill the process. As we plan to make D3E freely available, the technical startup costs for other journals will be much reduced. As an action research project, JIME's editors (the authors) and its infrastructure are funded by their respective institutions.

JIME's domain of concern is not a hard science such as physics, but one in which softer criteria for acceptance/rejection must be negotiated. A lot of work is exploratory, since there is not yet theoretical apparatus in place to shape much work. Interdisciplinarity adds further confusion, making JIME's more discursive review debates a suitable vehicle. In addition, educational multimedia researchers are generally techically literate with good Internet connections, which eases the process of adoption. We recognise, however, that not all fields are accustomed to discourse of this sort, or tempo. Only scholars who know their communities can judge whether interactive, open peer review could be deployed fruitfully.

Finally, at least in the U.K., research funding bodies now officially recognise e-journals as peer reviewed publications, although of course every journal must carve its niche in the hierarchy of journals on the basis of its content and rigour of review. We hope that the transparency of JIME's review process and review participants will be an asset for authors using JIME articles as part of their promotion and tenure cases, as any uncertainties about the quality and rigour of the review process can be immediately examined.

The above factors would be important to consider when weighing the potential of JIME-like e-journals in other fields. It should be borne in mind that the infrastructure could be used to support more conservative review models, e.g. using closed peer review, with or without the right of reply by authors and/or reviewers, with optional filtering of all postings via a moderator. The core computer literacy required is familiarity with point-and-click Web pages, and it also helps if readers are familiar with the idea of threaded discussions. Other skills develop in time, such as learning how to insert a URL to a comment in another thread (copy and paste), or (for editors) learning how to set up and facilitate a review discussion.

JIME as action research

JIME is a longitudinal action research project, studying the potential of the Internet for scholarly publishing and discourse through a real e-journal. We have devoted most of this article to describing its practical design and operation, but elsewhere we consider JIME from a variety of research perspectives, which the interested reader may wish to pursue. The emphasis on discourse derives from the notion of documents as social artifacts that accrue meaning through the discourse they provoke (Brown and Duguid, 1996; Brown and Duguid, 2000). Secondly, it is not realistic to expect end user communities (in our case, editors, reviewers and readers) to change instantly their accustomed ways of working in order to effectively use new technologies. A process of technology use-mediation is needed by which more expert users model ways of using the new technology for newcomers to see and gradually adopt (Sumner et al., 2000). Thirdly, the Web demonstrated how a powerful technology like the Internet could remain inaccessible to non-technical communities until suitable user interfaces were devised. D3E, the Digital Document Discourse Environment of which JIME is just one example, is based on several user interface design principles derived from pre-Web research into computer-mediated design argumentation (Sumner and Buckingham Shum, 1998a). Fourthly, D3E occupies a particular position in a broader design space of technologies to support collective interpretation through structured discourse (Buckingham Shum and Selvin, 2000). Finally, as it starts to be used by various universities, D3E offers interesting possibilities for managing student critiques and discussion of course texts (Buckingham Shum and Sumner, 1998).

++++++++++

Conclusion

Early photography looked like still life painting. Early cinema pointed a camera at the stage. Only gradually did new applications and genres evolve. As we enter the twenty-first century, scholarly publishing is in transition, so it is no surprise to find that the vast majority of e-journals are digital replicas of paper, with technology adding value primarily as a rapid production and dissemination medium. In this article we have described an e-journal which exploits the Web's potential as a medium for discourse through the co-evolution of the peer review model, underlying internet technologies and Web user interfaces. As with any new journal, JIME must still work to carve a niche in its own field, but the underlying approach and technologies are generic, discipline-specific factors notwithstanding.

The infrastructure is being laid rapidly for the global dissemination of scholarly documents and data. There will be an ocean of information available to scholars, highlighting all the more the need for tools to assist in filtering and interpreting it. Peer review is the fairest and most established means that scholars have devised for determining significance. We suggest that established review processes should not be unquestioningly transplanted from the papyrocentric environment in which they were grown into the online medium. The dynamics can be very different, and electronic journals should embrace this transitional phase as the chance to explore more effective forms of scholarly discourse and quality control.End of article

 

About the Authors

Simon Buckingham Shum is a Lecturer at the Open University's Knowledge Media Institute. His research concerns the human dimensions to technologies for knowledge-intensive communities, including scholarly publishing and organizational memory. His background is in human-computer interaction, design cognition, ergonomics and psychology.
E-mail: sbs@acm.org
Web: http://www.kmi.open.ac.uk/sbs

Tamara Sumner is an Assistant Professor with the Center for LifeLong Learning and Design (L3D) at the University of Colorado at Boulder. Her research interests include human-computer interaction, design support systems, and educational technology. Lately, she has been applying these interests towards the design of innovative scholarly technologies such as e-journals and digital libraries.
E-mail: sumner@colorado.edu
Web: http://www.cs.colorado.edu/~sumner

Acknowledgements

We are indebted to Diana Laurillard of the Open University for original support in developing JIME's vision, the Knowledge Media Institute for hosting it, Mike Wright for developing JIME's infrastructure and D3E-HyperNews, Malcolm Story for developing the D3E Publisher's Toolkit, and Gary Li for developing D3E-Phorum.

Notes

1. JIME's document discussion interface is an example of the Digital Document Discourse Environment (D3E), originally developed at the Open University, and now co-developed with the University of Colorado, Boulder, with a view to the creation of an open source community. The D3E Publisher's Toolkit is a Java application that transforms an HTML submission into the frames-based Web interface shown in Figure 1, with active contents and citations, linked to a structured discussion area. D3E can currently generate files for tailored versions of two freely available Web discussion systems: HyperNews, written in Perl and requiring a Unix/Linux Web server such as Apache, and Phorum, written in PHP and using the MySQL database, running on a range of Web servers, on any platform. A version also exists which will generate a structured discussion space for any URL, making it simple to add basic D3E functionality to any other environment. Look and feel is customised through HTML templates. It is D3E policy to reuse, integrate with, and disseminate open source code as far as possible.

Future developments could include integration with document submission and archiving software, such as the recently released Open Archives Initiative's Eprint Server software. The result would provide research communities with a free, open source solution to document dissemination and online peer review (cf. the CERN Workshop on OAI Archives and Peer Review).

2. An example of an embedded multimedia extract (using the Macromedia Shockwave Web browser plugin) can be found in the article at www-jime.open.ac.uk/96/1, specifically at www-jime.open.ac.uk/96/1/demo/seeing.html

3. An example of a downloadable multimedia demonstration can be found in the article at www-jime.open.ac.uk/97/2

4. An example of a guided tour to a system with audio commentary can be found in the article at www-jime.open.ac.uk/97/2, specifically at www-jime.open.ac.uk/97/2/demo/demo.html

5. An example of an educational Web site integrated into a JIME article can be found at www-jime.open.ac.uk/98/11, specifically at <www.cse.unsw.edu.au/~jas/poli/sample/

6. An example of embedded video data can be found in the article at www-jime.open.ac.uk/98/7, specifically at www-jime.open.ac.uk/98/7/repenning-98-7-10.html#4.2.2

7. An example of a submission in three hypertext formats can be found at www-jime.open.ac.uk/00/ingraham

8. Examples of established paper journals evaluating, or experimenting with, open peer review are the British Medical Journal (Smith, 1999) and the Australian Journal of Medicine www.mja.com.au/public/papers/papers.html

9. Psycoloquy is an interdisciplinary electronic journal publishing target articles and peer commentary in psychology and related disciplines. It solicits short articles from researchers seeking peer feedback on early work (Harnad, 1996); www.cogsci.soton.ac.uk/psycoloquy

10. International Journal of Human-Computer Studies, volume 47, number 1, pp. 1-222, Special Issue: World Wide Web Usability, S. Buckingham Shum and C. McKnight (editors); www.hbuk.co.uk/ap/ijhcs/webusability

References

B. Anderson, 1983. Imagined Communities: Reflection on the Origin and Spread of Nationalism. London: Verso.

J.S. Brown and P. Duguid, 1996. "The Social Life of Documents," First Monday, volume 1, number 1 (May), at firstmonday.org/issues/issue1/documents/ http://dx.doi.org/10.5210/fm.v1i1.466

J.S. Brown and P. Duguid, 2000. The Social Life of Information. Boston: Harvard Business School Press.

S. Buckingham Shum and C.E. McKnight, 1997. "World Wide Web Usability: Special Issue," International Journal of Human-Computer Studies, volume 47, number 1, pp. 1-222, at www.hbuk.co.uk/ap/ijhcs/webusability http://dx.doi.org/10.1006/ijhc.1997.0132

S. Buckingham Shum and A.M. Selvin, 2000. "Structuring Discourse for Collective Interpretation," Distributed Collective Practices 2000: Conference on Collective Cognition and Memory Practices, Paris, 19-20 September 2000), at www.limsi.fr/WkG/PCD2000

S. Buckingham Shum and T. Sumner, 1998. "New Scenarios in Scholarly Publishing and Debate," In: M. Eisenstadt and T. Vincent (editors). The Knowledge Web: Learning and Collaborating on the Net. London: Kogan Page, pp. 135-151, and at www.kmi.open.ac.uk/knowledgeweb

Dearing Report Discussion Site, 1997. Knowledge Media Institute, The Open University, U.K.

S. Fish, 1980. Is there a Text in this Class? The Authority of Interpretive Communities. Cambridge, Mass: Harvard University Press.

P. Ginsparg, 1996. "Winners and Losers in the Global Research Village," Conference on Electronic Publishing in Science, UNESCO: Session: Scientists' View of Electronic Publishing and Issues Raised, Paris, 19-23 February 1996), at arXiv.org/blurb/pg96unesco.html

F. Godlee and T. Jefferson, in press. Peer Review in Medicine. London: BMJ Books.

S. Goldbeck-Wood, 1999. "Evidence on Peer ReviewScientific Quality Control or Smokescreen?" British Medical Journal, volume 318, pp. 44-45, and at www.bmj.com/cgi/content/short/318/7175/44 http://dx.doi.org/10.1136/bmj.318.7175.44

S. Harnad, 1996. "Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals," In: R. Peek and G. Newby (editors). Scholarly Publication: The Electronic Frontier. Cambridge Mass.: MIT Press, pp. 103-108.

JIME: Journal of Interactive Media in Education. Knowledge Media Institute, Open University, at www-jime.open.ac.uk

T. Kuhn, 1996. The Structure of Scientific Revolutions. Third edition. Chicago: University of Chicago Press.

F.W. Lancaster, 1985. "The Paperless Society Revisited," American Libraries, , volume 16, number 8, pp. 553-555.

B. Latour, 1987. Science in Action: How to Follow Scientists and Engineers Through Society. Milton Keynes, U.K.: Open University Press.

Learning.Org: Developing an Organisational Strategy for Learning Technologies (see D3E Web Forum). Knowledge Media Institute, Open University, U.K., at www.kmi.open.ac.uk/learning.org

S. Lock, 1985. A Difficult Balance: Editorial Peer Review in Medicine. London: Nuffield Provincials Hospital Trust.

A. Marion and E.L. Hacking, 1998. "Educational Publishing and the World Wide Web," Journal of Interactive Media in Education, volume 98, number 2, at www-jime.open.ac.uk/98/2

H.B. O'Connell, 2000. "Physicists Thriving with Paperless Publishing," Invited Address, American Association for the Advancement of Science, Annual Meeting and Scicnce Innovation Exposition, 17-22 February, Washington, D.C., and at arxiv.org/abs/physics/0007040

R.D. Pea, 1999. "New Media Communication Forums for Improving Education Research and Practice," In: E.C. Lagemann and L.S. Shulman (editors). Issues in Education Research: Problems and Possibilities. San Francisco: Jossey Bass, pp. 336-370.

R. Smith, 1999. "Opening Up BMJ Peer Review," British Medical Journal, volume 318, pp. 4-5, and at www.bmj.com/cgi/content/full/318/7175/4 http://dx.doi.org/10.1136/bmj.318.7175.4

T. Sumner and S. Buckingham Shum, 1998a. "From Documents to Discourse: Shifting Conceptions of Scholarly Publishing," Proceedings, CHI 98: Human Factors in Computing Systems, pp. 95-102, and at www.kmi.open.ac.uk/tr/papers/kmi-tr-50.pdf

T. Sumner and S. Buckingham Shum, 1998b. "A Toolkit for Publishing Web Discussion Documents: Design Principles and Case Studies," Proceedings, APCHI 98: Asia Pacific Computer Human Interaction Conference, Kanagawa, Japan, 15-17 July.

T. Sumner, S. Buckingham Shum, M. Wright, N. Bonnardel, and A. Chevalier, 2000. "Redesigning the Peer Review Process: A Developmental Theory-in-Action," Proceedings, COOP 2000: Fourth International Conference on the Design of Cooperative Systems, Sophia Antipolis, France, 23-26 May, and at www.kmi.open.ac.uk/tr/papers/kmi-tr-96.pdf

S. van Rooyen, F. Godlee, S. Evans, N. Black, and R. Smith, 1999. "Effect of Open Peer Review on Quality of Reviews and on Reviewers' Recommendations: A Randomised Trial," British Medical Journal, volume 318, pp. 23-27, and at www.bmj.com/cgi/ijlink?linkType=ABST&journalCode=bmj&resid=318/7175/23 http://dx.doi.org/10.1136/bmj.318.7175.23

S. van Rooyen, F. Godlee, R. Smith, S. Evans, and N. Black, 1998. "The Effect of Blinding and Masking on the Quality of Peer Review: A Randomised Trial," Journal of the American Medical Association, volume 280, pp. 234-237. http://dx.doi.org/10.1001/jama.280.3.234

WAME, 1994-2001. International Congresses on Peer Review in Biomedical Publication (World Association of Medical Editors), www.wame.org


Editorial history

Paper received 22 January 2001; accepted 29 January 2001.


Contents Index

Copyright ©2001, First Monday

JIME: An Interactive Journal for Interactive Media, by Simon Buckingham Shum and Tamara Sumner
First Monday, volume 6, number 2 (February 2001),
URL: http://firstmonday.org/issues/issue6_2/buckingham_shum/index.html