This study reports on user based research on the annotating and linking prototype components of the Open Journal Systems of the Public Knowledge Project. The study describes the features of the components and provides results based on an analysis of both quantitative and qualitative data. The paper concludes by stating design implications of the research, and outlines future research based on development of the prototype.
Previous annotation studies
The role of linking and link typing
Discussion of results
Linking and link typing
Conclusion and future research
Annotation of digital content is a popular development focus for Webbased content and communication systems. Increasingly these tools are aimed at taking advantage of recent Webbased technologies that seek to enhance the interactive experience of users while reading online text. Annotation is no longer simply a means to attach notes to content, but a way to interact with the content, with the additional promise of interacting with other readers (and authors).
In keeping with this larger effort, we are currently working toward adding an annotation and linking capability to the Open Journal Systems (OJS), a Webbased, open access, scholarly journal publishing and management application developed as part of the Public Knowledge Project at the University of British Columbia (Willinsky, 2005). An important initiative within the OJS is the development of a set of ‘Reading Tools’ with the purpose of enhancing the online reading experience and improving the quality of critical engagement with online journal articles published within the system. Toward the goal of increasing critical engagement we have created an annotation and linking prototype module that we are currently testing with users. The study presented here reports on these user studies and provides directions for future development of these tools.
Previous annotation studies
Annotations have historically been personal records of reading and interpretation of what has been read. They also often serve as shared records of work and opinion within specific communities (Constantopoulos, et al., 2004). In a print environment, annotation and its associated forms (e.g. highlighting) are common activities (Olsen, 1994). Healy (1999) argues that annotation engages readers in thinking, making mental connections, and meaningfully linking ideas.
More specifically, reading scholarly journal articles is viewed as a highly directed activity (Adler, et al., 1998; Bishop, 1998; Marshall, et al., 1999; Kaplan and Chisik, 2005). Readers actively pursue meaning, carrying on an active mental dialogue with the writer. Previous research on the annotation habits of students and professionals demonstrated that the readers annotations are highly goaloriented (Wolfe, 2000; Marshall, et al., 1999; Marshall and Brush, 2004).
Recent research also illustrates the importance of annotations in writing, including discussion of use and form (Marshall and Shipman, 1997; Marshall, 1998; Ovsiannikov, et al., 1999), the influence of annotation on readers (OHara, et al., 1998), and their role in scholarly communication (Furuta and Urbina, 2002). Schilit, et al. (1998) enumerate some further advantages of the direct annotation of documents:
Convenience: Since annotations are directly integrated with the reading material, writers do not have to swap to a different tool to make statements about the content. Annotating the document in situ does not interrupt the flow of reading, minimizing forced divided attention (Schraefel, et al., 2002).
Immersion in document context: Annotations contain more information because they occur within the context of a document (rather than as isolated information object in).
Visual search: Annotations stand out visually in a document, allowing readers to easily scan for them.
Reviews of existing studies show that support for annotation of electronic documents promotes and facilitates active reading (Marshall, 1998; Schilit, et al., 1998; Wolfe, 2000; Golovchinsky and Marshall, 2000; Bernheim Brush, et al., 2002; Kaplan and Chisik, 2005). Obendorf (2003) carried out an observational study that compared different use patterns of annotation for the task of active reading. The support for active reading offered by traditional paperandpencil vs. two existing annotation tools for the Web was examined, and Obendorf concluded that simple annotation tools should be made available to facilitate personal annotation, and the participants expected the system to create additional value out of their notes.
Reviews by Wolfe (2002) and Reeve and Han (2005) showed that the new developments in annotation technologies tend to vary in input devise, interface, base text, anchor, storage, searching and filtering, and specialized behavior. The survey by Wolfe (2002) describes a range of currently available and developing technologies for creating and presenting annotations, glosses, and other comments on digital documents. The potential application of these tools for providing feedback to student writers, supporting extended group discussions around digital texts, and facilitating research and readingtowrite tasks are discussed.
Surprisingly, there are few studies that report results of user interactions with electronic annotation systems. Van Oostendorp (1996), for example, found that users accuracy in an electronic annotation condition was equivalent to the annotation accuracy using traditional paper and pencil methods. Marshall, et al. (1999) and Marshall and Brush (2004) investigated the relationship between personal annotations people make when they are discussing material online. Shipman, et al. (2004) distinguished between meager markers and happy highlighters in their study of law students use of the Xlibris ebook. Although not involving the use of an electronic system directly, Fu, et al. (2005) investigated users preferences for Web annotation tools.
The role of linking and link typing
The present study also builds on previous research (Kopak, 1999; 2000; 2002) that asserts that functional link typing, which describes the purposeful relationship between two nodes of information created by a link, can add significant functionality to electronic document environments by enabling navigation through the semantic space of a created hypertext.
For this study we have built a hypertext capability into OJS that enables users to create hypertext links between content segments within individual journal articles. We propose to investigate the efficacy of this hypertext capability as well as typed relationships through their instantiation as Reading Tools. The linking tool created seeks to leverage the functional relationships established in earlier research toward increasing the level of coherence between fragments of journal articles being read and annotated within the system. Subsequently, readers become authors by creating functionally meaningful relationships between these fragments. Movement through the articles within the system can be recalled and retraced with navigation guided by the purpose and role that associated information has in facilitating comprehension. The tool will enable the identification of related information (from the perspective of the reader), and the ability to type the relationship (e.g. illustrates, defines, generalizes, exemplifies, etc.).
For the study reported on here, 15 participants (12 females, 3 males) were recruited within a Library and Information Science department via postings on relevant email lists. Their academic backgrounds were varied, with individuals holding degrees in English Literature (3), Linguistics (2), as well as Psychology, Law, Political Science, Engineering, Mathematics, Archeology, Drama, Art History, and Music prior to entering their current degree program. Three of the participants were doctoral students, two were faculty, with the remainder being Masters students. While all were experienced Web users, only 30 percent reporting previous experience with some form of online annotation. If given the choice of reading from paper, or reading online, 75 percent of the participants stated that they would prefer reading documents in paper form. Forty percent of participants had some familiarity with the concept of hypertext having read at least two articles previous to the study.
In advance of the scheduled session, each participant was provided with a paper copy of an article on the topic of hypertext. They were asked to familiarize themselves with the contents of this article prior to the day of the session so that they could begin the session with a sufficient level of topical awareness. At the beginning of each session, participants were given a brief tutorial on how to use the prototype annotation and linking tools of the OJS. Subsequently, each session proceeded across three major task sections: a general reading, a directed reading, and a confirmatory reading section. In this report only the results from the General and Directed Readings tasks will be discussed.
In the first instance, participants were asked to browse through the four articles that were presented to them (one of which was an electronic version of the one provided in advance). In the directed reading section, participants were asked to select a specific topic (e.g. metaphor, disorientation, etc.) that they found interesting from the general reading section and to focus on this topic in reading the four articles.
Although not reported on here we can describe the final task section. Participants were asked to both review their own annotations and links and to review a selection of annotations prepared prior to the session. Comments were solicited about which of their own links and annotations they would be willing to share with other readers of the article, and also about which of the prepared annotations they found useful in helping them to better understand the articles.
Participants were encouraged to use the annotation and linking tools during each of the first two tasks. While interacting with the articles, participants were asked to verbalize their thoughts. In addition to the audio recording of participants verbal protocols, a screen capture program (Morae) was used to log screen events and to provide playback of screen activities for later analysis. Each session concluded with a semistructured interview focusing on the users experience in using the prototype components, and with a short questionnaire asking users to rate the usefulness of each of the major components of the tools.
Figure 1: OJS with Reading Tools and Annotation margin.
As stated, the study utilized a multimethod approach for data collection, including thinkaloud (verbal protocols), screen capture, observation notes and semistructured interviews, and pre and postsession questionnaires. Studies of this kind involve balancing concerns of rigor and realism (Marshall and Brush, 2004). Following Marshall and Brushs example, participants were given a uniform task, corpus, and technology, yet were also allowed to go about the task flexibly. Although the results of some quantitative analyses are reported, the results are largely qualitative. Efforts were made to triangulate the data where feasible and meaningful. Qualrus was used to analyse the qualitative data, and SPSS for the quantitative data.
The next section describes the findings related to participants use of the annotation and linking tools.
Discussion of results
The highlighting tool was designed in a manner that would be familiar to most users of online text, i.e. the mouse button is held down and the cursor drawn across the segment to be highlighted. As implemented, the highlighting feature mapped well onto what they were familiar with in paper environments. Generally, the only issue participants had with the highlighting was the perceptual difficulty of distinguishing between highlights that overlapped, and the inability of the tool to clearly associate marginal notes with corresponding overlapping highlights.
This particular issue of distinguishing between highlights reflects a larger concern of participants with their ability to choose which highlights they wanted to see, and when they wanted to see them. At times it became apparent that participants found the highlighting distracting, especially in the Directed Reading task where the focus of the participant changed from location to interpretation and the function of the highlight (to signal an important text fragment) was no longer required. As pointed out by Ray :
Well, I find when Ive highlighted something and I go back to read it for a different purpose, the highlighting sometimes is distracting. So, I would want to be able to turn it on and off, maybe even by session, just to say, I dont want to see my [highlights] as I read through this again, but I then might want to see them all.
Table 1 shows the frequency of highlighting alone, and the frequency of highlighting for the purpose of notemaking for each of the two reading tasks. A note is defined as a descriptive text element entered into the annotation area. For the purpose at hand, no distinction is made between longer notes and shorter notes (including the incidence of one word notes). Further note that in the implementation of the annotation component, a text segment must be highlighted before a note can be made.
Concerning the nature of the content segment highlighted in each annotation event, we can see from Table 1 that there is a preference for text segments at sentence length or less in the highlights. To a degree this reflects the typical structure of journal articles where names and diagrams appear less frequently. Nevertheless, participants appear focused on the marking of text segments in both conditions. The average extent of the highlighted segments (General and Directed Reading tasks combined) was 18 words, although this mean is negatively skewed by the data from three happy highlighters (Shipman, et al., 2004) who averaged 46 words per highlight (overall mean adjusted = 11 words). Likewise, four participants accounted for over threequarters (76.4 percent) of the number of complete paragraphs highlighted.
Furthermore, during the general reading task (when participants were familiarizing themselves with the content of the articles) there was a greater incidence of highlighting headings (110 instances or 27 percent of the total number of highlights made). Of these 110 instances of annotations, 46 (42 percent) were highlighting only and 64 (58 percent) were highlights made for the purpose of making a note. In the case of headings that were annotated with notes, participants characteristically described their purpose as summarizing what the section was about. As Melanie stated, Ill just highlight this heading, and Ill summarize [via a marginal note] the whole thing.
From Table 1 we see that the proportion of text segments highlighted when compared to the incidence of highlights accompanied with text annotations is noteworthy. In both the general and directed reading conditions the majority of annotations made were of the latter with 67 percent of the highlighting being made for the purpose of notemaking in the General Reading condition, and nearly three quarters (74 percent) made in the Directed Reading condition. One might expect that in the Directed Reading condition there would be a higher incidence of notemaking as participants were reading more for the purpose of understanding. In the General Reading condition one might alternatively have expected a higher proportion of the annotations to be highlighting only as participants sought to look for important passages to come back to, or use the highlighting tool to mark phrases, key words etc. that they thought significant, and that they would commit to memory for later use. As alluded to earlier, this latter expectation is borne out by the higher level of headings marked in the General Reading condition where participants were forming more general notions about the coverage and content of each article overall.
It is also observed that there is little to distinguish between the number and length of the notes in the two conditions. The researchers had expected that the number and length of notes would be greater during the Directed Reading task. In the General Reading task it was anticipated that participants would exhibit annotation behaviour associated with skimming and gist gathering resulting in more highlighting than notemaking, and also with fewer words per note made. Alternatively, the Directed Reading task was expected to result in more frequent notemaking, and with longer notes as comprehension and interpretation became the focus. No statistically significant difference between either the number [t = .93, p (twotail) = .34] or length [t = -.94, p (twotail) = .37] of the notes in the general and directed reading conditions was found. Figures 1 and 3 visually present the variation in the number and size of notes made by each participant in both the General and Directed Reading conditions.
With regard to the high incidence of notemaking in both conditions, we might conjecture that this is an effect of the test situation where participants were anxious to engage with the tools regardless of the specific task. Furthermore, concerning the size of the notes, the effect of placing the text annotation in the margin of the document (as opposed to a separate window, for example) can limit the extent of the note. This is especially so if it is desired that the complete note stay in close proximity to the associated content. It is reasonable to conclude that these were mitigating factors.
Nevertheless, there is an alternative interpretation that may have useful design consequences, at least concerning the number of text annotations. In our observations of the participants, and in the analysis of the verbal protocols, it became evident that participants were using marginal notes during the General Reading task to collocate content for future reference. In other words, notes were written to tag content that could be reviewed thematically later in the Directed Reading task. For example, all instances of metaphor or disorientation were labeled as such in the marginal note, and these instances were revisited during the Directed Reading task when the participant would focus on that particular concept or theme. Penelope described this as looking for patterns:
I want to go back and revisit ... and be able to find all the annotations that, while I was reading them, they came up. I think whats very handy about this as Im going through it now is ... the collocation function you can perform afterwards in terms of all the times you put context, context, context, to a background, right? Its like subject headings ... but youre not linking it outside the document yet, youre just within the document youre trying to find patterns.
This particular explanation only accounts for the unexpected higher frequency of text notes in the General Reading condition. What we can say from the analysis is that the creation of text notes for this purpose resulted in the use of individual words or short phrases to describe the content rather than longer phrases and sentences. The use of the text notes to describe content, and to use this capability as a form of advance organizer, though, seems a potentially useful one. In the section on link typing to follow we will elaborate on the value of this feature, and provide further indication of its efficacy.
The lack of variability in note length between the two reading conditions may be more likely a result, alluded to earlier, of the physical limitations of the prototype. In addition to the constrained workspace inherent in locating text notes in the margin of the document, participants also looked for greater functionality in the processing of the notes. Participants expressed the desire to edit their notes beyond the rudimentary capability offered by the prototype system including: the ability to add bulleted items, the ability to visually differentiate between original notes and additions or emendations (e.g. colour coding), and the ability to search their annotations. More capability in the editing of text annotations is interpreted as a desire to further a more active reading strategy where users can more easily integrate the reading and writing function.
Adrian, a university professor, remarked on the potential offered by this in writing research papers:
Yes, this is something that allows the reader to react to the text. And I think that really Im seeing here as Im going through this is that you can start imagining how ... Im trying to imagine how a lot of the writing of the paper would be ... it would be a much ... the transition from reading to writing would be different. Because instead of, you know, making notes and making notes and making notes, and then rearranging the notes in a word processing document, sort of moving ideas around and moving sections around, its like we know that it actually belongs there. I would be inclined to spend more time going through here, and actually doing the writing through the annotations. So, doing a lot of the organization of the essay or the paper through the annotation system. And then translate it at a much later time into like, a word processor document, where youre actually laying it out.
Similar to their concern with the distraction caused by highlighting, participants also expressed the need to control which text annotations they wanted to see at any given point in their reuse of an article. As Geoff noted:
I think that it would be useful if I could turn individual ones off, and then maybe a global one that turns them all off. Maybe there is some that I dont really want to delete, but I dont want them sitting there either. Or maybe it doesnt reflect my current thinking as it did at one point. So, it might be an idea to be able to toggle off certain annotations rather than having to delete them.
This quote generally reflects a desire to see only annotations that are appropriate to the specific context of the reader. It was noted repeatedly by participants that the meaning and interpretation of certain of their own text annotations were unclear to them even by the end of the research session. Characteristically, these kinds of text annotations, in addition to the more idiosyncratic notes made in the General Reading task, were viewed as dispensable, and that some easy way of deleting these annotations would be an asset. Nevertheless, most of the annotations were viewed as a form of intellectual capital that could have meaning in a specific context to be revisited at a future point in time. In this case, an easy way of hiding specific annotations was desired.
Linking and link typing
Perhaps the most novel of aspect of the prototype annotation components are the linking and link typing tools. Our goal in developing these tools was to instantiate a hypertext component into OJS that would take advantage of the ability of linking to provide and additional form of annotation. If we consider one of the functions of annotation is to provide a means to connect a related piece of information to a content segment, we can accomplish this as well by linking directly to an existing piece of information rather than creating a new one. Additionally, we sought to enable readers to add value to the link by attributing the functional nature of the relationship between the content segment and the linked item. We did so by providing a list of link types characterizing these kinds of relationships, i.e. the linked item defined, illustrated, compared, etc. the content segment, and that could be easily applied to the link.
A demonstration of the linking and link typing tool was included as part of the tutorial provided to each participant at the beginning of each participant session. Participants were instructed to create links, and type the link where they thought it was appropriate and judged useful.
Table 2 shows the frequencies of use of both the linking and typing components in both the General and Directed Reading tasks. The first row indicates the number of links created in both tasks, while the second row indicates the number of times the link types were used.
First, we note that the use of both linking and link typing was greater in the General Reading condition. This was counter to expectations in that linking was conjectured to be representative of meaning relationships more characteristic of the Directed Reading task than the General Reading task. Creating a link demands, at least conceptually, a certain understanding of the content items at both ends of the link. Furthermore, it would appear to require some consideration of the nature of the relationship between the content items before attributing a link type. Neither of these requirements is indicative of the purposes of the General Reading task where gist and place marking are more typical.
As before, we might conjecture that this outcome is a result of the procedure, where participants were keen to try out the linking and typing capability in the early stages, but increasingly relied on the more familiar components as the session progressed. Furthermore, the heavier cognitive load required to create these relationships may have resulted in a certain attenuation in motivation. Although the link creation was accomplished through a relatively straightforward procedure, the overall condition was likely compounded by the relative unfamiliarity participants had with the idea of link creation in an operational system like OJS. Typical of the challenges participants had in using the linking component were those encountered by Joyce:
Maybe I would try linking this [Roland Barthes] to the first article where they talked about readers and authors, because this Roland Barthes has a lot to do with that kind of theory. Now Im trying to remember how to do that? I have forgotten how to do that. So if I want to link this name to that point, how can I do that again? So, if I want to link this name to the readers and authors part in the first article, how do I do that again? Yes, the asterisk. Now Im going to find that paragraph again, which is at the beginning with the Information Creation [Linking]. I am not entirely sure that this is going to be very helpful.
Although participants had some difficulty, it largely appeared due to the lack of an appropriate mental model of the linking function. Even though participants understood the idea of hypertext, and were actually reading about it during the course of the session, they had no clear image of how the linking would work in the larger context of use. Once a scenario of use became clearer to them, attitudes became more positive. For example, after the uncertainty expressed by Joyce in the previous excerpt she became quite expressive of its virtue:
Oh! That works! Perfect! That is actually a really cool function. Lets try to see it. Oh, yah! Then you can click the back button? Ok, that works. Thats really a good function, I think. Because, it is difficult, sometimes to keep track ... . I know I read a lot of articles, and then I think of, oh, this relates to this point of that article, if I am writing an essay and I want to tie these two points together, and then I keep reading and then I cant find the first article, or I dont remember which article it was, or its lost in the pile of paper on my desk, and this actually you have the first article, and you make a link, at least you can find the article that links to it. At least you have a starting point.
A more specific design issue of the linking component is the manner in which the linked node is presented to the reader. When a link is engaged, the current prototype returns the target node to the same browser window, displacing the anchor node. Although this mimics a typical Webbased browser interaction it was problematic in the context of journal article use in OJS. Both observation and verbal protocols indicated disorientation on the part of participants when the target node replaced the anchor node. Typically, participants were expecting the target node to appear in a separate window so that the original text remained on the screen with the anchor text. As George stated:
I think its that, for a minute I think Im looking at the Landow [the source article], and I dont know why I would be discombobulated by it, but I think that I would like to see them continue to be ... side by side for Im still in the text with Landow, Im not thrown out of the text with Landow. I have ... I can still see that [Landows article] as well as the link, because its the relationship, you know, so its ... you know, hypertext allows you to go on these different pathways but in some ways for not getting disoriented or lost, you would like to be able to still see it in relation to the place it ... came from.
This sense of physical relationship (the sidebysidedness) appears important in sustaining the context created by the link itself. The points of departure and arrival (Landow, 1989) are important rhetorically and semantically, and having each point proximally represented appears efficacious in this context of use. Concerning context, it is also noted that the signaling of arrival carries additional responsibility in orienting the reader to what they are seeing once it arrives. It is important to recall that the prototypelinking tool enables the user to create a link between information components at the paragraph level. As such, it is likely that the section of the target document presented to the user is lacking any significant identifying information. The exception to this would be the instance in which the target paragraph appeared near the beginning of the article and contained title and author information. As John suggests,
So it would be nice to have some kind of identifying information when I jump to another page. I would also like to have some kind of identifying information on the target end, in terms of what that article is, whos talking, maybe just some identifying information. Now were just jumping to that other article right in the middle of it, so theres no sense of necessarily how to relate it other than through creating the link. So Id like to have some way of giving some identifying descriptive information of the source document, so I can say, oh yes, this is Burbules talking now. Im not sure how that would be represented.
Returning to Table 2, we see that the third and fourth row report the number of link types used with, and without links. We are presented here with the curious observation that the large majority (74 percent) of the link types used did not reference an actual link. Recall that in the section on text notes it was observed that participants used the annotation as a collocation device, essentially applying a form of subject heading to the content to be used to identify and gather similar themes within the article. Interestingly, participants continued this strategy by using the link types to describe text content rather than a link relationship. For example, identification of unfamiliar words and terms within the content was an important aspect of participants interaction with the text content in both the General and Directed Reading conditions. Fully 209 instances within the verbatim transcripts were coded where the participant expressed a desire to know the meaning of a particular term. Of these 209 instances, participants used the definition link type provided in the dropdown link type menu on 20 occasions to indicate that a definition was needed. Overall, link types were used on 57 occasions to describe content rather than a link relationship, compared to the 20 times they were employed for their intended use.
Overall, the use of the link types for their intended purpose was problematic. As stated, almost three quarters of the instances in which they were used were to describe article content in the form of a text annotation. Of those using the typing facility for the designed purpose there were consistent challenges concerning the interpretation of the meaning of the type definitions themselves. Whereas a link type like definition or example was relatively straightforward to apply when a link to a definition in another article was made, often the application of link types such as background and comment were perceived as ambiguous and participants were unsure about their interpretation of them.
Yeh, the meaning is here [mouses over and show the definition of Comment]. And I think, again, Oh, Im sure there is something here that would describe the relationship that I was trying to do here. But, maybe the relationship I want is just different. The list here, they seem like specific. It was hard. It really was just a general connection. I dont know. I think, it may be that social tagging might work better, because you could maybe define the relationship yourself. But, it also makes sense to have a list. This looks like, if you can agree upon what Comment means. If someone comes back, and sees Comment, you have the same idea of what it is. I am not really sure on that. So, maybe you just define everything better, or just more thoroughly. This could work very well.
Table 3 reports the results from a postsession questionnaire that asked participants to rate the usefulness of the major components of the annotation and linking tool. As part of the larger study undertaken, we gathered data, as well, on participants attitudes and opinions about sharing their annotations and links with others. These others included colleagues in a work group, and the public at large. Although these data are not discussed here, the overall ranking of the tools by audience is considered useful. Highlighting was viewed as more personal due largely to its idiosyncratic nature and lack of cues to establish motivation and meaning outside of ones specific task context. Text annotation (notemaking) was viewed as appropriate across all levels with 93 percent viewing notetaking as useful, or very useful in the personal role, 73 percent in the workgroup, and 73 percent for the general public. Linking was judged appropriate across all groups with link typing seen as useful across all groups, although viewed as slightly more applicable for workgroups and the public. Perhaps the overall view on the efficacy of the various components of the annotation and linking tool was express by Jack:
Yeah, I like that theres only a few tasks [tools]. The key ones like highlighting and annotating, by far, are the ones that I would be most frequently doing. And the linking, thats nice, but I wouldnt be doing it as much. And the link typing, I dont know. I dont know if I would do that, to be honest. I really am intrigued by the concept, but I dont know if I would actually do it on a regular basis. But its worth a whirl, just as something thats there to ... I would find that intriguing as a user to see that that was there, and may eventually, once I gained comfort with the environment, and got the annotating that was part of my regular practice, then eventually I might add on the link typing. But I wouldnt be doing it the first day. Whereas the highlighting and annotating, absolutely. That would be right in there. That would totally be something Id be doing. I can see that theres a bit of addictive quality to it, and it helps with onscreen reading because provides some kind of interactive engagement that helps draw me into the text. And thats a good thing.
As Webbased information services continue to provide even more advanced communicative capabilities we can expect users engagement with digital materials to increase. As Marshall and Brush (2004) have suggested we can also expect, as part of this, that an increasing number of individuals will read and annotate these materials as easily as they would paper documents. In order for this to occur, annotation of digital materials must increase the level of critical engagement with these digital artifacts by more ably leveraging the capabilities and functionalities inherent in the medium, and especially of those in a networked environment. Our study provides some evidence toward increasing this level of engagement by reporting on user interactions with an annotation and linking prototype for an online scholarly journal system.
Our findings suggest a number of design recommendations that can be usefully employed in a variety of online annotation systems. These focus on three major areas of development: highlighting, notemaking, and linking and link typing.
It is quite apparent that the highlighting tool is an integral part of an annotations system. It not only offers an obvious means of drawing attention to a text segment, but also provides a means to placemark and signal items for future reference. Whether it is required as a means to initiate a text annotation is less certain. Although often accompanying a note, highlighting is not necessary for its creation. In our study, highlighting was on occasion viewed as an impediment. For example, in the creation of summary notes for a section, participants often did not know what to highlight in order to create the annotation, as the note referenced a complete section or several paragraphs rather than a phrase or sentence.
Equally, a means of disambiguating overlapping highlights is required. For example, differentiating by using different colour highlights was suggested. This might also be carried forward to enable additional capacity of the highlight to represent different purposes, e.g. blue representing quotable quotes, and yellow definitions, based on some degree of personalization.
Perhaps most important is the capability to easily remove highlighting, either permanently, or temporarily. This should be relatively finegrained, allowing selective removal of identified highlights. It is not only the case with highlighting, but with text annotations as well that the readers wish from time to time to remove selected items, or to view a completely clean copy of the original document.
Concerning text annotations, there is some uncertainty about the manner in which these kinds of annotations might be created and displayed. For the prototype described here, the choice was made to use marginalia as opposed to a separate annotation window. One might reduce this to a matter of aesthetics, but there are important functional considerations. The marginalia approach limits the size of a text annotation to an amount that keeps the larger part of the annotation in proximity to the content segment to which it is related. This also limits the amount of writing that can occur within the text annotation and is prerequisite to the desire for more text editing functionality. Conversely, the intimacy and direct spatial relationship of the marginal annotation has its advocates. As one participant remarked:
So, maybe it might be an idea to have the annotations in a separate window. But, you know, its kind of nice to have the sense of fixity in the relationship. The windows are sort of separate information. In a way, this [marginalia] is more intimate, a comment, its more directly connected to the original point in the article ... . I mean, having your annotations in one big view isnt very helpful if you dont know what you were referring to when you made the annotation.
Of consequence as well, is the use of text annotations as advanced organizers and as aids to building internal representations of the texts, and in the restructuring and collating of information from the text (OHara and Sellen, 1997). This was found to be an important feature of text annotations where patterns, or threads, are identified within a document either according to topical similarity or argument. In conjunction with greater text editing functionality, the text annotation would further increase engagement with the system by providing the building blocks for the creation of new texts.
Further to context, it is important as well to provide for identifying information for incoming text nodes. At minimum, it would seem prudent to represent a text node with at least the title and author visible within the browser window.
Although the linking and link typing components of the prototype were limited in their ability to engage participants, they do have an important role. Linking enables the creation of a text beyond the text and integrates individual documents into new wholes according to the purposes of the reader/author. Perhaps what are required are better representations of the network created by the interlinked corpus. Generally, it was found that participants lacked a model that would engage them in the use of a personal hypertext, perhaps limited by the simple relationship between two documents. In terms of the link typing, it is clear that a more compact and unambiguous set of types is required. Also required is an efficient procedure for users to more easily integrate the types with their own annotations. Participants did recognize the merit of typing, but also recognized that they could become more meaningful if they could be customized with the addition of their own text annotations.
Conclusion and future research
This study represents one of the few available on user interactions with a system employing annotation and linking components. As part of OJS, the prototype annotation and linking module proved successful in enabling an interaction component to be tested in an operational environment with wellstructured content.
Overall, the prototype proved successful, at least in its ability to confirm aspects of the system that worked, and to provide direction for future research and development. Some of the major issues to be addressed were described in the previous section.
One important area of future development concerns the role of social interaction in an environment like that offered by OJS. The prototype largely enables annotation and linking for personal use. We are currently carrying out a study that investigates more specifically, the role of social input to content description and navigation through both the annotation and linking components. We are investigating the kinds of annotations, for example, that readers would and would not find useful in an environment where one could contribute their annotations to a public information space.
Most interestingly, we are also investigating the social construction of hypertexts via the linking tool available in the prototype. Our interest here is in the ability to construct navigable planes through an information space presented in a collection of journal articles (or other content) contained in a system like OJS through social identification of link paths.
About the authors
Rick Kopak is a faculty member at the School of Library, Archival and Information Studies at the University of British Columbia in Vancouver, B.C. He is currently coinvestigator on the Public Knowledge Project. He received his PhD from the Faculty of Information Studies at the University of Toronto.
ChiaNing Chiang is a PhD candidate in the School of Library, Archival and Information Studies at the University of British Columbia in Vancouver. She is currently working on her dissertation entitled A Multidimensional Approach to the Study of Online Annotation.
The study was funded in it entirety by a grant from the Social Science and Humanities Research Council of Canada (SSHRC, #41020030880).
1. Names are fictional, but are referenced to actual participants.
A. Adler, A. Gujar, B.L. Harrison, K. OHara, and A. Sellen, 1998. A diary study of workrelated reading: Design implications for digital reading devices, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 98), pp. 241248.
A.J. Bernheim Brush, D. Bargeron, J. Grudin, and A. Gupta, 2002. Notification for shared annotation of digital documents, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis), pp. 8996.
A.P. Bishop, 1998. Digital libraries and knowledge disaggregation: The use of journal article components, Proceedings of the Third ACM Conference on Digital Libraries (Pittsburgh), pp. 2939.
P. Constantopoulos, M. Doerr, M. Theodoridou, and M. Tzobanakis, 2004. On information organization in annotation systems, Lecture Notes in Artificial Intelligence, number 3359, pp. 189200; and at http://www.ics.forth.gr/isl/publications/paperlink/ioas.pdf, accessed 2 October 2007.
X. Fu, T. Ciszek, G. Marchionini, and P. Solomon, 2005. Annotating the Web: An exploratory study of Web users needs for personal annotation tools, Proceedings of the 68th Annual Meeting of the American Society for Information Science & Technology (Charlotte); at http://ils.unc.edu/annotation/publication/Fu_etal_ASIST05.pdf, accessed 2 October 2007.
R. Furuta and E. Urbina, 2002. On the characteristics of scholarly annotations, Proceedings of the Thirteenth ACM Conference on Hypertext and Hypermedia (Hypertext 2002, College Park), pp. 7879; and at http://www.csdl.tamu.edu/cervantes/pubs/HT02cp.pdf, accessed 2 October 2007.
G. Golovchinsky and C.C. Marshall, 2000. Hypertext interaction revisited, Proceedings of the 11th ACM Conference on Hypertext and Hypermedia (San Antonio), pp. 171179, and at http://www.fxpal.com/publications/FXPALPR00082.pdf, accessed 2 October 2007.
J.M. Healy, 1999. Endangered minds: Why children dont think — and what we can do about it. New York: Simon and Schuster.
R. Kopak, 2002. Link typing in hypertext: Defining conceptual attributes, Proceedings of the Canadian Association for Information Science (Toronto), pp. 215222.
R. Kopak, 2000. A taxonomy of link types for use in hypertext, Ph.D. thesis, University of Toronto.
R. Kopak, 1999. Functional link typing in hypertext, ACM Computing Surveys, volume 31, number 4, pp. 1622. http://dx.doi.org/10.1145/345966.346005
N. Kaplan and Y. Chisik, 2005. Reading alone together: Creating sociable digital library books, Interaction design and children: Proceeding of the 2005 Conference on Interaction Design and Children (Boulder), pp. 8894.
G.P. Landow, 1989. The rhetoric of hypermedia: Some rules for authors, Journal of Computing in Higher Education, volume 1, number 1, pp. 3964. http://dx.doi.org/10.1007/BF02942605
C.C. Marshall, 1998. Toward an ecology of hypertext annotation, Proceedings of Ninth ACM Conference on Hypertext and Hypermedia (Pittsburgh), pp. 4049; and at http://csdl.tamu.edu/~marshall/ht98-final.pdf, accessed 2 October 2007.
C.C. Marshall and A. Brush, 2004. Exploring the relationship between personal and public annotations, Proceedings of the Joint Conference on Digital Libraries JCDL 07 (Tucson), pp. 349357; and at http://csdl.tamu.edu/~marshall/112-marshall.pdf, accessed 2 October 2007.
C.C. Marshall and F.M. Shipman, 1997. Effects of hypertext technology on the practice of information triage, Proceedings of the ACM Hypertext 97 Conference (Southampton), pp. 124133; and at http://csdl.tamu.edu/~marshall/ht97.pdf, accessed 2 October 2007.
C.C. Marshall, M.N. Price, G. Golovchinsky, and B.N. Schilit, 1999. Introducing a digital library reading appliance into a reading group, Proceedings of ACM Digital Libraries (Berkeley), pp. 7784; and at http://csdl.tamu.edu/~marshall/ht97.pdf, accessed 2 October 2007.
H. Obendorf, 2003. Simplifying annotation support for realworldsettings: A comparative study of active reading, Conference on Hypertext and Hypermedia: Proceedings of the Fourteenth ACM Conference on Hypertext and Hypermedia (HT 03; Nottingham), pp. 120121.
K. OHara, and A. Sellen, 1997. A comparison of reading paper and online documents, Proceeding of CHI 97 (Atlanta), pp. 335342, and at http://sigchi.org/chi97/proceedings/paper/koh.htm, accessed 2 October 2007.
K. OHara, F. Smith, W. Newman, and A. Sellen, 1998. Student readers use of library documents: Implications for library technologies, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 98, Los Angeles), pp. 233240; and at http://www.hpl.hp.com/personal/Kenton_Ohara/papers/Library_CHI.pdf, accessed 2 October 2007.
J. Olsen, 1994. Electronic journal literature: Implications for scholars. London: Mecklermedia.
I.A. Ovsiannikov, M.A. Arbib, and T.H. McNeill, 1999. Annotation technology, International Journal of HumanComputer Studies, volume 50, number 4, pp. 329362. http://dx.doi.org/10.1006/ijhc.1999.0247
L. Reeve and H. Han, 2005. Survey of semantic annotation platforms, Proceedings of the 2005 ACM Symposium on Applied Computing (Santa Fe), pp. 16341638.
B.N. Schilit, G. Golovchinsky, and M.N. Price, 1998. Beyond paper: Supporting active reading with free form Digital Ink annotations, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 98, Los Angeles), pp. 249256.
M.C. Schraefel, Y. Zhu, D. Modjeska, D. Wigdor and S. Zhao, 2002. Hunter gatherer: Interaction support for the creation and management of withinWebpage collections, Proceedings of the 11th International World Wide Web Conference (Honolulu), pp. 172181.
F. Shipman, M. Price, C.C. Marshall, and G. Golovchinsky, 2004. Identifying useful passages in documents based on annotation patterns, Proceedings of the 7th European Conference on Research and Advanced Technology for Digital Libraries (Trondheim), pp. 101112; and at http://www.fxpal.com/publications/FXPAL-PR-03-216.pdf, accessed 2 October 2007.
H. Van Oostendorp, 1996. Studying and annotating electronic text, In: J.F. Rouet, J. Levonen, A. Dillon, and R.J. Spiro (editors). Hypertext and cognition. Hillsdale, N.J.: Lawrence Erlbaum Associates, pp. 137148.
J. Willinsky, 2005. Open Journal Systems: An example of open source software for journal management and publishing, Library HiTech, volume. 23, number 4, pp. 504519. http://dx.doi.org/10.1108/07378830510636300
J. Wolfe, 2002. Annotation technologies: A software and research review, Computers & Composition, volume 19, pp. 471491. http://dx.doi.org/10.1016/S8755-4615(02)00144-5
J. Wolfe, 2000. Effects of annotations on student readers and writers, Proceedings of the Fifth ACM Conference on Digital Libraries (San Antonio), pp. 1926.
Copyright ©2007, First Monday.
Copyright ©2007, Rick Kopak and ChiaNing Chiang.
Annotating and linking in the Open Journal Systems by Rick Kopak and ChiaNing Chiang
First Monday, Volume 12 Number 10 - 1 October 2007