The visibility of Wikipedia in scholarly publications
First Monday

The visibility of Wikipedia in scholarly publications by Taemin Kim Park



Abstract
Publications in the Institute of Scientific Information’s (ISI, currently Thomson Reuters) Web of Science (WoS) and Elsevier’s Scopus databases were utilized to collect data about Wikipedia research and citations to Wikipedia. The growth of publications on Wikipedia research, the most active researchers, their associated institutions, academic fields and their geographic distribution are treated in this paper. The impact and influence of Wikipedia were identified, utilizing cited work found in (WoS) and Scopus. Additionally, leading authors, affiliated institutions, countries, academic fields, and publications that frequently cite Wikipedia are identified.

Contents

Introduction
Brief introduction to Wikipedia
Literature review
Data collection
Data analysis and discussion
Summary and conclusions

 


 

Introduction

No one denies that Wikipedia is now a highly used, albeit controversial, information source. Wikipedia has become increasingly an important tool for “fact–checking” (Kniffel, 2008) as well as a topic of research because of its convenient access on the Web, its coverage, and the nature of large–scale collaborative work, among other reasons. According to WorldCat (24 August 2010), Wikipedia has been a topic of more than 50 theses and dissertations worldwide and has been a subject of more than 200 monographic publications.

The purpose of this study is to explore the extent of Wikipedia’s presence in scholarly publications in Web of Science (WoS) and Elsevier’s Scopus databases. The Institute for Scientific Information (ISI), publisher of WoS, asserts that it contains the world’s leading citations from multidisciplinary coverage of over 10,000 high–impact journals in the sciences, social sciences, and arts and humanities, as well as international proceedings coverage for over 120,000 conferences. WoS covers Science Citation Index Expanded indexing over 6,650 major journals, Social Science Citation Index containing over 1,950 journals and Arts and Humanities Citation Index for 1,160 of the world’s leading arts and humanities journals [1]. Scopus states that it contains 18,000 titles from more than 5,000 international publishers, including 16,500 peer–reviewed journals in addition to about 1,200 open access journals, 600 trade publications, 2,350 book series, and 3.6 million conference papers among others [2]. Some differences in WoS and Scopus databases should be noted. The scope and types of publications included in WoS and Scopus differ and this should be taken into account in understanding the search results and interpretations. It is clear that WoS covers journals more selectively while Scopus covers a much higher numbers of conference papers. A recent study on the journal title overlap between WoS and Scopus databases reported that about 45 percent of titles in Scopus are not covered in WoS, while 16 percent of titles in WoS are not covered in Scopus (Gavel and Iselid, 2008).

 

++++++++++

Brief introduction to Wikipedia

Wikipedia: About page defines Wikipedia as a multilingual, Web–based, free–content encyclopedia project based on an openly editable model. Anyone can contribute and edit the Wikipedia articles. Users can contribute anonymously, or under a pseudonym, or with their real identity. The page history view (revision history or edit history) includes a list of the page’s previous revisions, including date and time, the user name (or IP address) and edit history. However, Cohen (2009) reported that the English Wikipedia added an imposing layer of editorial reviews on articles about living people declared no longer available in the openly editable mode. Since its inception in 2001, Wikipedia has published 17,000,000 articles. There are currently 91,000 active contributors, and Wikipedia is now available in 270 languages. The English Wikipedia alone includes more than three million articles, 23 million pages, more than 446 million edits and is attracting 79 million visitors monthly on the Internet, as of January 2011. Wikipedia was founded as an offshoot of Nupedia, founded by Jimmy Wales and officially launched on 15 January 2001.

Among approximately three million articles in the English Wikipedia, there are about 3,194 (about 0.1 percent) featured articles. Featured articles represent the best articles which, according to Wikipedia’s featured list criteria, have undergone a thorough review process by Wikipedia’s editors to meet the highest standards for usefulness, completeness, accuracy, neutrality and style. A featured article has a small bronze star icon on the top right corner of the article’s page. Citing a study conducted by researchers at the Carnegie Mellon University and Palo Alto Research Center, the Wikipedia’s site lists the most frequently covered topics — Culture and the arts (30 percent), Biographies and persons (15 percent), Geography and places (14 percent), Society and social sciences (12 percent), History and events (11 percent), among others. Spoerri (2007) examined the popularity of topics in Wikipedia and found the most popular Wikipedia pages were related to entertainment and sexuality. Popular pages appeared to be related to search engines, especially Google. The site reports that the growth of the English Wikipedia in terms of new articles and contributors reached a plateau in early 2007. Landgraf (2009) also reported a reduction in Wikipedia’s growth. Kopytoff (2011) reported on the celebration of Wikipedia’s tenth anniversary and mentioned plans to increase the number of foreign language articles by opening an office in India, then possibly Egypt and Brazil. Plans also include the recruitment of a wider range of contributors — more women, elderly, and, to add more graphical content, museum experts.

 

++++++++++

Literature review

Reviews in the library and information science literature indicated that Wikipedia itself has increasingly become a subject of research from diverse academic disciplines due to its exceptional scale and utility (Medelyan, et al., 2009). The concept of “information quality (IQ),” incorporating collaboration, evolving debates, and process as assurance, was studied using Wikipedia as an example (Stvilia, et al., 2008). A Wikipedia entry, Wikipedia: Academic studies of Wikipedia (http://en.wikipedia.org/wiki/Academic_studies_about_Wikipedia) reports a partial list of academic writings about Wikipedia reported in journal articles, and conference proceedings among other formats and the Academic studies about Wikipedia page includes some Wikipedia research in peer–reviewed publications.

A question about Wikipedia’s quality and reliability as an information source has been one of the most frequently investigated research topics. In an evaluation of Wikipedia as a reference source, applying the classic reference evaluation criteria — purpose, authority, scope, audience, cost, and format — Danny P. Wallace and Connie Van Fleet (2005) concluded that Katz’s criteria for reference sources do not stand up well to Wikipedia. A comparison of Wikipedia and other encyclopedias in historical entries revealed that Wikipedia’s accuracy was 80 percent compared with 95–96 percent accuracy in other sources (Rector, 2008). A special report by the prestigious weekly journal Nature (Giles, 2005) raised commentary from Encyclopedia Britannica. Nature’s investigation, based on 42 science entries, found that both Wikipedia and Britannica contained numerous errors, but the difference in accuracy was not great. The average inaccuracy rate in Britannica was about three per article while Wikipedia contained about four. The number of edits, collaborators, and edit patterns were studied in relation to article quality. Wilkinson and Huberman (2007) compared the number of edits and contributors to the 1,211 “featured” articles to the same number of other articles to test the correlation between number of edits and article quality. They concluded that Wikipedia article quality appeared to increase on average as the number of collaborators and number of edits increase. Revising patterns — the total number of editors, the number of edits, and the number of major and minor edits — in a sample of two groups of articles were studied to determine their relationship to article quality (Poderi, 2009). The study reported that not every contribution had the same weight and major edits were not necessarily contributing to article quality. The role of main editors differed in the two groups of article. The articles in the group with a high presence of main editors tended to become featured articles more easily. Other aspects of quality such as Wikipedia’s biased coverage and lack of cited sources were identified as “Wikipedia risks” (Black, 2008). Nielsen (2007) examined about 30,368 outbound links in Wikipedia’s science entries. Although the number of linked citations to scholarly literature was small compared to the number of citations found in scientific journals, Wikipedia showed a slight tendency to cite articles in high–impact ISI journals. For example, the largest number of citations were to Nature, Science and the New England Journal of Medicine in the sample studied.

Coverage of philosophers in the twentieth century listed in Wikipedia and in two other widely used online resources was compared for data regarding their birth date, gender, national and disciplinary backgrounds. This study found that Wikipedia contained more entries for living and ‘minor’ philosophers than traditional resources (Elvebakk, 2008). The semantic coverage of the English Wikipedia was studied and represented in terms of baseline statistics for articles, subject categories, and the top 10 authors (Holloway, et al., 2007).

Use of Wikipedia is on the rise. While some university professors have banned using Wikipedia as a research source (Cohen, 2007), use of Wikipedia was promoted using epistemic values. Fallis (2008) argued that there were good epistemic consequences of using Wikipedia as a source of information by illustrating some empirical examples. Epistemic values such as power, speed, immediate availability, wiki technology, the wisdom of crowds, and Wikipedia policies were noted as outweighing the deficiencies in the reliability of Wikipedia. Despite controversies, use of Wikipedia by academic communities has been expanding. More positive responses to Wikipedia have been reported from academic libraries. For example, libraries at the University of Washington, University of North Texas, and Wake Forest University, among others, have decided to participate in Wikipedia by editing, adding links, or writing new articles (Lally and Dunford, 2007; Pressley and McCallum, 2008). Lim’s (2009) survey on college students’ use of Wikipedia also showed that students use it as a source for quick fact–checking and for finding background information. Student’s perceptions regarding information utility and their positive emotions toward Wikipedia were related to their usage level. Use of Wikipedia in college class room has been reported. One of Wikipedia’s recent projects, Public Policy Initiative (http://outreach.wikimedia.org/wiki/Public_Policy_Initiative), became a teaching resource in some universities [3]. For example, five universities — Georgetown, George Washington, Harvard, Indiana University and Syracuse — were invited to work on editing articles on the policy–related entries in Wikipedia to improve the article quality.

Citation counts in scholarly publications have been frequently used as an important tool — to assess the relative scholarly impact of research, diffusion of new research ideas, to study journals, individual researchers, and to identify maps of scholarly communication across scientific specialties and so on (Meho and Sugimoto, 2009). Cronin and Shaw (2007) used bibliometric tools to identify Kling’s intellectual impact and network using his publications, his cited works, and acknowledgment data. Others studied citing behaviors and motivations of citers besides scientific impact (Bornmann and Daniel, 2008). For the citation counts, ISI databases (such as WoS), Scopus and Google Scholar are the most often used tools. ISI’s three citation databases were the only comprehensive citation data source until Elsevier’s Scopus and Google Scholar were launched in 2004. In a paper comparing the citation counts provide by WoS, Scopus and Google Scholar for articles from the Journal of the American Society for Information Science and Technology, Bauer and Bakkalbasi (2005) conclude that Google Scholarr likely retrieves traditional journal articles which are also possibly covered by WoS and Scopus in addition to unique citations. However, the coverage of scholarly publications was the least in Google Scholar.

 

++++++++++

Data collection

The visibility of Wikipedia in scholarly communications was examined based on the following questions:

  1. How many times Wikipedia has been a topic of research in scholarly publications covered in WoS and Scopus databases?
  2. Who are the contributors most often engaged in doing research about Wikipedia?
  3. What are these authors’ institutional affiliations?
  4. Which publications have published studies on Wikipedia most often?
  5. Which academic fields are engaged in studying Wikipedia most frequently?
  6. How often Wikipedia has been cited in the scholarly publications covered in WoS and Scopus databases?
  7. Who cites Wikipedia most often?
  8. Which publications cite Wikipedia most frequently?
  9. Which academic fields cite Wikipedia most often?
  10. Authors from which institutions most frequently cite Wikipedia in their publications?

Two types of data were collected to examine the visibility of Wikipedia in scholarly publications. The presence of Wikipedia in scholarly publications was assumed if a study’s major topics include “Wikipedia,” or “Wikipedia” has been used in their references. A search in WoS using Wikipedia in the topic OR title field was conducted in January 2011 to find the number of records for which a publication’s topic is Wikipedia. A truncated search was used to match any variations and to achieve a more comprehensive search result. In the same way a search in Scopus in the title, abstract, or keyword fields was conduced. There were 291 records in WoS and 1,455 in Scopus with topics including Wikipedia. Scopus allows a search conducted beyond its own databases by providing Web searching options. Yet, this research was limited to Scopus alone as it includes only peer–reviewed publications.

The search result displays typical citation information including author(s), title (document), source title, its volume and number designation, pagination (if available), and publication year. From the search in WoS, all search results were selected to display the list of publications with a main topic on Wikipedia and to refine the result using ISI analysis tools. These analysis tools allow the search results to be sorted by ranked order for a selected field (e.g., author, institutional name, country, etc.). For example, Brendan Luyt and Oded Nov have published most frequently on Wikipedia in scholarly publications covered by WoS. A search result in Scopus displays ranked lists of each field, for example, by source title, author name, publication year, affiliation, subject area, document type, etc. Advanced search features in Scopus were utilized for more precise and comprehensive searching. For example, a search combined with the field “affilcountry” (United States or US) displays publication output by researchers affiliated with institutions located in the United States. An “affilorg” (Hong Kong) brings additional research output by researchers affiliated with institutions in Hong Kong. An advanced search combined with “subjarea” (comp) shows the number of documents categorized as computer science.

To examine aspects of Wikipedia’s impact, a search for “cited work = Wikipedia*” was conducted in WoS. In a “Cited Reference Search”, all references in the WoS databases that cite Wikipedia were retrieved. The search result listed cited author(s), cited work (Wikipedia), year (if available), and the number of times cited for a specific article. There were 340 records citing Wikipedia in WoS. Once the search is executed, all entries which cite Wikipedia are selected, then the search is finished. One should note that the number of citing articles on the “Cited Reference Search” page and the number listed in the “Times Cited” count on the results page after finishing the search might differ depending on the scope of one’s institution’s subscriptions to various databases within WoS. The “Times Cited” count on the results page are counted from all the databases in WoS: Science Citation Index Expanded, Social Science Citation Index, Arts and Humanities Citation Index, Conference Proceedings Citation Index–Science, and Conference Proceedings Citation Index–Social Sciences and Humanities. For example, if an institution has a subscription to the Science Citation Index Expanded and Social Science Citation Index, Arts and Humanities Citation Index but not the Conference Proceedings Citation Index, the number of citing articles on the “Cited Reference Search” page may be smaller. In addition, the result may be influenced by one’s subscription periods. If an institution has access to a limited time period such as from 2005 to the present, the result would probably be smaller.

Larry Dossey and Brendan Luyt cited Wikipedia most often in their scholarly publications as noted in WoS. In a similar way, a search for “refsrctitle = wikipedia*” was conducted in Scopus for publications with the source title (Wikipedia) in references. There were 3,339 records citing Wikipedia as source titles in references in Scopus. All search results were downloaded into an MS Excel file for data analysis.

 

++++++++++

Data analysis and discussion

Country productivity

There were, as of January 2011, a total of 1,746 publications in WoS and Scopus for the period 2002 to 2010, which contained research about Wikipedia. The number should be taken with caution due to overlapping coverage of publications between WoS and Scopus as noted earlier. Furthermore, these numbers may change as the coverage of publications in the WoS and Scopus databases is updated.

To achieve a more precise measurement of research production by country, Hong Kong was searched separately and added to China’s production for data analysis. China’s production included three additional publications from Hong Kong in WoS. Likewise, for the United Kingdom, additional searches were conducted in WoS for England, Scotland, Wales, and Northern Ireland. Four publications from Scotland were added to the United Kingdom’s total. In a similar manner, an additional country search for Hong Kong in Scopus added 18 more publications to China. The country name, United Kingdom, was used consistently in Scopus for all publications affiliated with that nation. Table 1 lists the most productive countries in Wikipedia research. The most productive countries were the U.S. and the United Kingdom in WoS and the U.S. and Germany in Scopus. The next most productive countries were China, France, the United Kingdom, Japan, Italy and the Netherlands in Scopus. The U.S. is far stronger in producing research on Wikipedia than any other country, accounting for about 22 percent of the publications in Scopus and about 37 percent in WoS.

 

Table 1: Research production on Wikipedia by country.
CountryNumber of publications in WoS
(percent)
CountryNumber of publications in Scopus
(percent)
United States107 (36.8)United States315 (21.6)
United Kingdom25 (8.6)Germany137 (9.4)
Germany22 (7.6)China
(including Hong Kong)
99 (6.8)
Canada13 (4.4)France69 (4.7)
Australia12 (4.2)United Kingdom65 (4.5)
China
(including Hong Kong)
12 (4.2)Japan64 (4.4)
France11 (3.8)Italy57 (3.9)
Italy9 (3.1)Netherlands57 (3.9)
Spain9 (3.1)Australia55 (3.8)
Netherlands9 (3.1)Spain50 (3.4)
Singapore9 (3.1)Canada42 (2.9)

 

Author productivity

Analysis of author productivity, based on the number of publications included in WoS and Scopus, indicated that a small numbers of authors created a number of publications. There were 291 publications with a total of 701 authors in WoS and 1,455 publications in Scopus with a total of 3,940 principal and collaborative authors with a research topic including Wikipedia. Multiple authorship was the norm. For example, one publication about Wikipedia research in Scopus was coauthored by 37 individuals.

Two individuals wrote 13 papers, another two researchers contributed 12 publications, and six published 10 items about Wikipedia. Altogether, 123 individuals wrote more than four publications on Wikipedia. The most highly productive 15 individuals, their affiliated institutions, countries, and the number of their publications are listed in Table 2. Individual researchers who developed the most frequent publications were affiliated with institutions located in Europe and Asian countries. Jaap Kamps at the University of Amsterdam (http://staff.science.uva.nl/~kamps/) and Gerhard Weikum of the Max–Planck–Institut für Informatik (http://www.mpi-inf.mpg.de/~weikum/) each wrote 13 articles dealing with, in some fashion, Wikipedia.

 

Table 2: Most highly productive authors in research on Wikipedia, based on Scopus.
NameAffiliationCountryNumber of publications
Kamps, J.University of AmsterdamNetherlands13
Weikum, G.Max–Planck–Institut für InformatikGermany13
Geva, S.Queensland University of TechnologyAustralia12
Nakayama, K.Osaka UniversityJapan12
Koolen, M.University of AmsterdamNetherlands10
Hara, T.Osaka UniversityJapan10
Kittur, A.Carnegie Mellon UniversityUnited States10
Ortega, F.Universidad Rey Juan CarlosSpain10
Nishio, S.Osaka UniversityJapan10
Sun, A.Nanyang Technological UniversitySingapore10
Demartini, G.L3S Research CenterGermany8
Jijkoun, V.University of AmsterdamNetherlands8
Milne, D.University of WaikatoNew Zealand8
Trotman, A.University of OtagoNew Zealand8
Witten, I.H.University of WaikatoNew Zealand8

 

Affiliated institution productivity

The majority of researchers on Wikipedia were affiliated with universities. The most productive 15 institutions are listed below in ranked order, in Table 3. Individual researchers affiliated with the University of Amsterdam, Nanyang Technological University and the Max–Planck–Institut für Informatik were the most productive in doing research on Wikipedia. These 15 affiliated institutions contributed 230 publications which were about 13 percent of the total publications in WoS and Scopus. Researchers affiliated with the Carnegie Mellon University and Indiana University were most active in research on Wikipedia in the United States.

 

Table 3: Most highly productive institutions on Wikipedia.
InstitutionNumber of papers
University of Amsterdam31
Nanyang Technological University23
Max–Planck–Institut für Informatik19
Queensland University of Technology17
Carnegie Mellon University17
University of Tokyo16
Indiana University15
University of Illinois at Urbana–Champaign12
Hewlett–Packard Laboratories12
Osaka University12
Shanghai Jiao Tong University12
University of Washington11
IBM Thomas J. Watson Research Center11
Georgia Institute of Technology11
Microsoft Research11

 

Academic fields which are most active in Wikipedia research

Table 4 describes 10 academic fields which are most active in Wikipedia research according to WoS and Scopus. Academic fields in this study were defined by these databases respectively. Scopus categorizes its content into 27 subject areas and WoS includes 251 subject areas. Certainly, computer science was the most productive. About 42 percent of Wikipedia research is produced from many areas in the computer science fields and about 26 percent from information and library science in WoS databases while in Scopus about 72 percent of research originates from the computer science category as defined by Scopus. The fields of mathematics, social sciences, and engineering are also highly productive. In Scopus an exceeding small portion of publications, about one percent of Wikipedia research output, derives from the arts and humanities. Note that a publication may be categorized in more than one subject category and thus the total number of publications may include duplication.

 

Table 4: Academic fields most active in Wikipedia research.
Academic fields
(WoS)
Number of publications
(percentage)
Academic fields
(Scopus)
Number of publications
(percentage)
Information science, Library science74
(25.6)
Computer science1,052
(72.3)
Computer science, Information systems73
(25.2)
Mathematics341
(23.4)
Computer science, Artificial intelligence24
(8.3)
Social sciences260
(17.9)
Engineering, Electrical and electronic19
(6.6)
Engineering199
(13.7)
Communications16
(5.5)
Biochemistry, Genetics and molecular biology109
(7.5)
Computer science, Theory and methods13
(4.5)
Decision sciences99
(6.8)
Education and education research13
(4.5)
Business, Management, Accounting85
(5.8)
Management13
(4.5)
Medicine44
(3.0)
Computer science, Hardware and architecture12
(4.2)
Agriculture and Biological sciences, Arts16
(1.1)
Multidisciplinary sciences10
(3.5)
Arts and humanities15
(1.0)
  Physics and Astronomy15
(1.0)

 

The leading publications reporting research on Wikipedia

Table 5 rank orders the 11 most productive publications on Wikipedia research in both WoS and Scopus. it appears that more research about Wikipedia has been published in conference papers and proceedings than in journal articles. As conference titles tend to vary frequently, more comprehensive searches for conference publications were conducted. Series such as Lecture Notes in Computer Science including the subseries Lecture Notes in Artificial Intelligence; Lecture Notes in Bioinformatics; Proceedings of the International Conference on Information and Knowledge Management; and, International Symposium on Wikis (with slightly variant titles) are the leading outlets for Wikipedia research. Lecture Notes in Computer Science is a major series; a WorldCat search retrieves more than 100,000 items. The International Symposium on Wikis’ Web site reports that it focuses on research and practice about wikis and open collaboration. Thus it appears to be a very appropriate venue for Wikipedia research. First Monday is also highly regarded in publishing Wikipedia research. Because of the coverage differences between WoS and Scopus, Wikipedia research is most often reported in journals in WoS and conference proceedings in Scopus. However, it is noteworthy that the Journal of American Society for Information Science and Technology (JASIST) appears on both lists. The top 11 publications produced about 20 percent of the Wikipedia research in WoS compared to about 37 percent in Scopus. It is interesting that Wikipedia research appears to be concentrated in a small number of publications as recorded in Scopus while scattered among a larger number in WoS.

 

Table 5: Leading serials publishing Wikipedia research.
Publication as reported in WoSNumberPublication as reported in ScopusNumber
Journal of American Society for Information Science and Technology14Lecture Notes in Computer Science, including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics292
Online Information Review6Proceedings of International Conference on Information and Knowledge Management61
Journal of Computer–mediated Communication5International Symposium on Wikis53
Journal of Web Semantics5International ACM SIGIR Conference on Research and Development in Information Retrieval29
BMC Bioinformatics4First Monday26
Computers in Human Behavior4AAAI Workshop Technical Report14
Electronic Library4Journal of American Society for Information Science and Technology14
Information Systems4Proceedings of AAAI National Conference on Artificial Intelligence13
Nature4Proceedings of ACM Conference on Human Factors in Computing Systems12
Information Retrieval4Proceedings of ACM Conference on Computer Supported Cooperative Work12
New Media & Society4International Conference on Knowledge Discovery and Data Mining11

 

Impact of Wikipedia

Citations to Wikipedia in scholarly publications were examined to test Wikipedia’s impact on scholarly communication. This effort attempted to identify those who cite Wikipedia most often, their affiliated institutions, associated fields, and geographic distribution.

Wikipedia was cited 3,679 times in the WoS and Scopus databases. The 11 researchers who cited Wikipedia most frequently in their scholarly publications were from eight countries. Saou–Wen Su, affiliated with the Lite–On Technology Corporation in Taiwan, cited Wikipedia in eight publications; Gerhard Weikum of the Max–Planck–Institut für Informatik cited Wikipedia in in seven publications. Table 6 lists individual researchers who cited Wikipedia most frequently in their papers as recorded by Scopus and WoS.

 

Table 6: Citation of Wikipedia by specific researchers.
NameCountryNumber
Su, Saou–WenTaiwan8
Weikum, GerhardGermany7
Boukerche, AzzedineCanada6
Ortega, FelipeSpain6
Ren, Y.United States6
Ros, L.France5
Hijazi, H.France5
González–Barahona, J.M.Spain5
Milne, DavidNew Zealand5
Witten, Ian H.New Zealand5
Wong, K.L.Malaysia5

 

Citations to Wikipedia by affiliated institutions

As illustrated below, authors affiliated with institutions in the U.S. appear to cite Wikipedia more often in their scholarly publications than authors in any other country. Researchers affiliated with Carnegie Mellon University, Georgia Institute of Technology, and Indiana University were most active in citing Wikipedia. The most highly citing affiliated institutions are rank ordered in Table 7. International researchers affiliated with universities in Asian countries — Nanyang Technological University, University of Hong Kong, Tsinghua University, and Chinese University of Hong Kong — cited Wikipedia most frequently. Nanyang Technological University, Carnegie Mellon University, Indiana University, and Tsinghua University were also listed among the 15 institutions which are most productive in Wikipedia research as well.

 

Table 7: Institutions whose researchers cite Wikipedia most frequently.
InstitutionNumber of citations
Carnegie Mellon University23
Georgia Institute of Technology19
Indiana University17
Institute of Electrical and Electronics Engineers (IEEE)16
Nanyang Technological University15
University of Hong Kong15
Purdue University15
New York University15
Tsinghua University15
Chinese University of Hong Kong14
Arizona State University14
University of California, Berkeley14
University of California, Los Angeles14

 

Citations to Wikipedia by country

Table 8 lists the number of citations to Wikipedia by country. Researchers from the U.S., China, United Kingdom, Germany and Canada most frequently cite Wikipedia according to Scopus while the U.S., United Kingdom, Canada, and Germany cite the most in the WoS database. For the United Kingdom’s total, an additional four citations from Scotland were added. Likewise, a combined search with “affilcountry” (Hong Kong) brought an additional 42 citations by researchers affiliated with institutions in Hong Kong in Scopus which were added into China. Scholars in the U.S., Germany, United Kingdom, China, and France were most active in generating research on Wikipedia while researchers affiliated in the U.S., United Kingdom, Germany, and China cited Wikipedia most often. American scholars are strong in both Wikipedia research and citing Wikipedia in their publications. However, a closer look reveals that U.S. scholars are more likely to cite Wikipedia than to actually produce research on Wikipedia itself. American scholars account for about 37 percent of published research on Wikipedia in WoS and 22 percent in Scopus whereas they produce 43 percent of the citations to Wikipedia in WoS and 27 percent in Scopus.

 

Table 8: Citations to Wikipedia by country.
CountryNumbers cited in WoS
(percent)
CountryNumbers cited in Scopus
(percent)
United States146
(43)
United States908
(27)
United Kingdom
(including Scotland)
19
(5.6)
China
(including Hong Kong)
212
(6.3)
Canada18
(5.3)
United Kingdom196
(5.8)
Germany14
(4.1)
Germany158
(4.7)
Australia11
(3.2)
Canada138
(4.1)
Singapore9
(2.7)
Australia116
(3.5)
China8
(2.4)
France78
(2.3)
Taiwan8
(2.4)
Japan75
(2.2)
Austria7
(2.1)
Italy63
(1.9)
France7
(2.1)
Netherlands62
(1.9)
Netherlands7
(2.1)
Spain46
(1.4)

 

Scholarly publications citing Wikipedia most often

The publications in WoS and Scopus which most cite Wikipedia were identified and are rank ordered in Table 9. Among the 22 publications that produced the most research about Wikipedia, four — namely Lecture Notes in Computer Science (with subseries), Proceedings of the International Symposium on Wikis, First Monday and Journal of the American Society for Information Science and Technology — also cited Wikipedia most frequently. Interestingly, in WoS the 10 most frequently citing publications contain about 12 percent of the total citations to Wikipedia while the 11 publications most active in producing Wikipedia research comprise about 20 percent of the publications about it. Likewise, in Scopus the 10 most highly citing publications contain only 12 percent of the relevant citations whereas the top 11 publications on Wikipedia research contain about 37 percent of pertinent publications. Wikipedia research is highly concentrated in a relatively few publications whereas citations to Wikipedia are scattered among a larger number of diverse publications in both WoS and Scopus. Thus, Wikipedia’s impact on scholarly communications appears to be stronger through citations to it rather than through publications about it.

 

Table 9: Publications in WoS and Scopus which cite Wikipedia most often.
Publications in WoSNumber of publications
(percent)
Publications in ScopusNumber of publications
(percent)
Lecture Notes in Computer Science11
(3.2)
Lecture Notes in Computer Science, including the subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics202
(6)
Journal of American Society for Information Science and Technology6
(1.8)
Proceedings pf the International Symposium on Wikis33
(1)
Publications of the Modern Language Association of America (PMLA)5
(1.5)
Proceedings of SPIE (International Society for Optical Engineering)30
(<1)
Computers & Security4
(1.2)
Proceedings of ACM International Conference Series28
(<1)
Explore: The Journal of Science and Healing4
(1.2)
Conference Proceedings of the American Society for Engineering Education (ASEE)26
(<1)
AAA — Arbeiten aus Anglistik und Amerikanistik3
(0.9)
First Monday21
(<1)
Journal of Universal Computer Science3
(0.9)
Proceedings of the International Conference on Information and Knowledge Management20
(<1)
Athletic Therapy Today2
(0.6)
Journal of the American Society for Information Science and Technology15
(<1)
Biochemistry and Molecular Biology Education2
(0.6)
Communications in Computer and Information Science14
(<1)
Clinical Orthopedics and Related Research2
(0.6)
Proceedings of the ACM Conference on Human Factors in Computing Systems13
(<1)

 

Academic fields citing Wikipedia most often

Table 10 displays the 12 academic fields which cite Wikipedia the most often as noted in WoS and Scopus. About 16 percent of the citations to Wikipedia originate from computer science fields, about 10 percent from information and library science, about six percent from literature, and about four percent from communications and engineering in WoS. In Scopus, about 42 percent of citations come from computer science, 24 percent from engineering, and another 21 percent from the social sciences. The computer science field displays both the highest proportion of Wikipedia research and citations to Wikipedia. The fields of engineering (24 percent), and medicine (14 percent) are quite active in citing Wikipedia in their publications. In contrast, 14 percent of Wikipedia research derives from engineering, and three percent from medicine. Mathematicians contribute a larger proportion of the Wikipedia research (23 percent) than the citations to it (11 percent). The proportions are nearly equal for social scientists who produce 18 percent of the Wikipedia research and 21 percent of the citations. In the arts and humanities the proportion of citations to Wikipedia (about four percent) is also greater than the proportion of research publications about Wikipedia (about one percent). Remember that a publication may be assigned to more than one subject category so citation counts by fields may include duplicates.

 

Table 10: Academic fields citing Wikipedia most frequently.
Academic fields identified in WoSNumber of citations
(percent)
Academic fields identified in ScopusNumber of citations
(percent)
Information science and Library science34
(9.9)
Computer science1,419
(42.5)
Computer science, Information systems27
(7.9)
Engineering797
(23.8)
Literature19
(5.5)
Social sciences711
(21.3)
Computer science, theory and methods17
(4.9)
Medicine483
(14.5)
Communications13
(3.8)
Mathematics366
(10.9)
Engineering, electrical and electronic13
(3.8)
Biochemistry, Genetics and Molecular biology183
(5.5)
Computer science, software engineering11
(3.2)
Arts and Humanities149
(4.5)
Education and Education research11
(3.2)
Business, Management and Accounting139
(4.2)
Law9
(2.6)
Physics and Astronomy139
(4.2)
Humanities, Multidisciplinary8
(2.3)
Material science109
(3.3)
Language and Linguistics8
(2.3)
Decision science102
(3.1)
Languages8
(2.3)
Nursing83
(2.5)

 

Wikipedia’s increasing visibility in scholarly communications

Scholarly research about Wikipedia apparently first appeared in the 3 June 2002 issue of First Monday, in paper entitled “Open source intelligence” by Felix Stalder and Jesse Hirsh [4] as well as in a 2002 article in Online entitled “Péter’s picks and pans review on Wikipedia” by Péter Jacsó [5]. Table 11 summaries the pertinent data about Wikipedia in WoS and Scopus from 2002 to 2010. As the Table 11 illustrates, research about and citations to Wikipedia in scholarly publications have steadily increased over time since its launch in 2001. Although citations to Wikipedia in WoS peaked in 2007, there is substantial evidence in citation patterns to demonstrate the significant impact of Wikipedia on scholarly communication over the past decade, corresponding to its increased use as an information resource.

 

Table 11: Research about Wikipedia and citations to Wikipedia, by year.
YearNumber of research publications identified in ScopusNumber of citations identified in ScopusNumber of research publications identified in WoSNumber of citations identified in WoS
20022021
20030400
2004339010
20051997724
2006803032270
20072094913381
20083405926557
20093908807648
20104129338649

 

Tables 12 and 13 present data about the types of publications that, respectively, write about and cite Wikipedia. Table 12 shows that in Scopus research about Wikipedia has been published predominantly in conference papers (63 percent), articles (26 percent), and review papers (three percent) among other formats, while in WoS it has been published more frequently in articles (65 percent), proceeding papers (nine percent), and editorial materials (seven percent). However, Wikipedia tends to be more highly cited in journal articles as shown in Table 13: 30 percent in Scopus and 70 percent in WoS. Only seven percent of the citations in WoS were to conference papers, contrasted to 31 percent in Scopus. In summary, the visibility of Wikipedia research is more prominent in conference and proceedings papers while citations to Wikipedia are more prevalent in journal articles.

 

Table 12: Types of publications publishing research about Wikipedia.
Document type identified in WoSNumber
(percent)
Document type identified in ScopusNumber
(percent)
Articles188
(65.0)
Conference papers921
(63.2)
Proceedings papers25
(8.7)
Articles385
(26.4)
Editorial material21
(7.2)
Reviews47
(3.2)
Book reviews15
(5.2)
Conference reviews46
(3.1)
Letters11
(3.8)
 
News items10
(3.4)
Other21
(7.2)
Other56
(3.8)

 

 

Table 13: Types of publications citing Wikipedia.
Document type identified in WoSNumber
(percent)
Document type identified in ScopusNumber
(percent)
Articles239
(70.3)
Conference papers1,046
(31.3)
Editorial material39
(11.5)
Articles995
(29.8)
Proceedings papers24
(7.1)
Reviews258
(7.7)
Reviews19
(5.5)
Editorial material54
(1.6)
Book reviews12
(3.5)
Short surveys21
(<1)
Letters6
(1.7)
Notes17
(<1)

 

 

++++++++++

Summary and conclusions

Since Wikipedia was launched in 2001, the number of research publications about Wikipedia and citations to Wikipedia has increased steadily. There were a total of 1,746 publications included in WoS and Scopus for the years 2002 to 2010.

Research about Wikipedia has been published most frequently by individual researchers who are affiliated with academic institutions in Europe and Asian countries — Netherlands, Germany, Australia and Japan. However, the largest proportion of research on Wikipedia has been contributed by scholars in academic institutions in the U.S. (about 37 percent in WoS and 22 percent in Scopus), followed by scholars from Germany, United Kingdom, and China. Researchers in universities are the major contributors to Wikipedia research. The University of Amsterdam in the Netherlands and the Max–Planck–Institut für Informatik in Germany were the most active in producing research on Wikipedia. Analysis by discipline shows that the most frequent contributors to Wikipedia research are computer scientists, information scientists, and mathematicians. For example, the Lecture Notes in Computer Science (with subseries) and Proceedings of the International Symposium on Wikis (with variant titles) have published more Wikipedia research than any other publications. Conference publications and journal articles are the major venues for reporting research on Wikipedia.

Wikipedia’s citation rates in scholarly publications have been consistently increasing. It was cited 3,679 times in the WoS and Scopus databases during the last nine years. Academic institutions are not only the major producers of Wikipedia research but also the major consumers that cite Wikipedia most often. The rate of citing was highest among scholars from the US, United Kingdom, Germany, and China. Wikipedia has been cited in more than 30 countries and by 306 institutions worldwide in WoS alone. Authors affiliated with academic institutes in the U.S. appear to cite Wikipedia most frequently. American scholars tended to cite Wikipedia to a greater extent than they published research about it. Researchers affiliated with Carnegie Mellon University, Georgia Institute of Technology, and Indiana University were the most active in citing Wikipedia in their publications. Scholars in the fields of computer science, information science and social sciences are the most active in citing Wikipedia. Interestingly, researchers in engineering and medicine cite more often than do research on Wikipedia, while researchers in mathematics more often write about Wikipedia than cite it. Arts and humanities also give more citations to Wikipedia than conduct research about it. Wikipedia research is most likely to be published in conference and proceedings papers, then journal articles along with other formats. However, citations to Wikipedia were more often found in journal articles followed by conference papers and then editorial materials. A few publications contain a high portion of Wikipedia research while citations were scattered in a wider range of publications. The breath of Wikipedia’s impact has stretched to authors in many fields and professional areas.

Reported numbers regarding the writing about and citing of Wikipedia should be taken carefully as they reflect only a snapshot provided by several databases. Since this research is based only on WoS and Scopus, publications included in these databases are mostly in English. Finally, book reviews, editorial material, letters, and news items (which constitute a significant portion of publications about Wikipedia in WoS) are not strictly speaking “research,” but they nevertheless are indicative of Wikipedia’s impact on the scholarly communication.

This research adds to our understanding of Wikipedia’s role in scholarship and reflects scholarly regard in some sense for a highly controversial yet well used resource on the Internet. This bibliometric study demonstrates Wikipedia’s visibility in the scholarly communication process — productivity of scholars, affiliated institutions, academic fields, and the geographic distribution of affiliated institutions, and the type of publications. The influence of Wikipedia on the scholarly community as indicated by citations was identified in the course of this research. Hence this paper sheds some light on trends regarding Wikipedia’s place in formal scholarship and demonstrates its growing visibility.

Recent involvement by higher education communities in Wikipedia implies Wikipedia’s potential to become not only a reliable resource but also a learning and teaching tool for students. Wikipedia’s plans to include more women and elderly as well as expanding international offices will bring balance and wholeness in content. As demonstrated in this study, active research on Wikipedia and citations to Wikipedia testifies to Wikipedia position as a rich resource. The increasing scholarly attention to Wikipedia suggests a growing acceptance of its credibility as a valid information resource.

This study is only a small step in demonstrating the visibility of Wikipedia in scholarly communication. Identifying major topics covered in scholarly publications about Wikipedia may be addressed in future research. Other issues — such as examining gender differences, co–author networks in Wikipedia research, and motivations for citing Wikipedia — could add further details on the utility of Wikipedia in scholarship. End of article

 

About the author

Taemin Kim Park, Ph.D., is an Associate Librarian of Indiana University Libraries and Adjunct Faculty in the School of Library and Information Science at Indiana University, Bloomington.

 

Acknowledgements

This research was conducted during the author’s research leave which was partially supported by Indiana University Libraries.

 

Notes

1. Web of Science (WoS) Web site, at http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/, accessed 24 August 2010.

2. Scopus Web site, at http://www.info.sciverse.com/scopus/, accessed 24 August 2010.

3. Steve Hinnefeld, 2010. “SPEA seminar producing policy articles for Wikipedia,&lrquo; at http://homepages.indiana.edu/web/page/normal/15707.html, accessed 21 July 2011.

4. Felix Stalder and Jesse Hirsh, 2002. “Open source intelligence,” First Monday, volume 7, number 6, at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/961/882, accessed 21 July 2011.

5. Péter Jacsó, 2002. “Péter’s picks & pans: Pans: Wikipedia,” Online, volume 26, number 2, pp. 81–82.

 

References

Kathleen Bauer and Nisa Bakkalbasi, 2005. “An examination of citation counts in a new scholarly communication environment,” D–Lib Magazine, volume 11, number 9, at http://www.dlib.org/dlib/september05/bauer/09bauer.html,accessed 25 April 2011.

Erik W. Black, 2008. “Wikipedia and academic peer review: Wikipedia as recognized medium for scholarly publication?” Online Information Review, volume 32, number 1, pp. 73–88.http://dx.doi.org/10.1108/14684520810865994

Lutz Bornmann and Hans–Dieter Daniel, 2008. “What do citation counts measure? A review of studies on citing behavior,” Journal of Documentation, volume 64, number 1. pp. 45–80.http://dx.doi.org/10.1108/00220410810844150

Noam Cohen, 2009. “New rules in wiki world,” New York Times (25 August), at http://query.nytimes.com/gst/fullpage.html?res=9501E5DC1430F936A1575BC0A96F9C8B63, accessed 20 August 2010.

Noam Cohen, 2007. “A history department bans citing Wikipedia as a research source,” New York Times (21 February), at http://www.nytimes.com/2007/02/21/education/21wikipedia.html, accessed 20 August 2010.

Blaise Cronin and Debora Shaw, 2007. “Peers and spheres of influence: Situating Rob Kling,” Information Society, volume 23, number 4, pp. 221–233.http://dx.doi.org/10.1080/01972240701444147

Beate Elvebakk, 2008. “Philosophy democratized? A comparison between Wikipedia and two other Web–based philosophy resources,” First Monday, volume 13, number 2, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2091/1938, accessed 30 August 2010.

Don Fallis, 2008. “Toward an epistemology of Wikipedia,” Journal of the American Society for Information Science and Technology, volume 59, number 10, pp. 1,662–1,674.

Ylva Gavel and Lars Iselid, 2008. “Web of Science and Scopus: A journal title overlap study,” Online Information Review, volume 32, number 1, pp. 8–21.http://dx.doi.org/10.1108/14684520810865958

Jim Giles, 2005. “Internet encyclopaedias go head to head,” Nature, volume 438, number 7070 (15 December), pp. 900–901, and at http://www.nature.com/nature/journal/v438/n7070/full/438900a.html, accessed 20 August 2010.

Steve Hinnefeld, 2010. “SPEA seminar producing policy articles for Wikipedia,&lrquo; at http://homepages.indiana.edu/web/page/normal/15707.html, accessed 21 July 2011.

Todd Holloway, Miran Bozicevic, and Katy Börner, 2007. “Analyzing and visualizing the semantic coverage of Wikipedia and its authors,” Complexity, volume 12, number 3, pp. 30–40.http://dx.doi.org/10.1002/cplx.20164

Péter Jacsó, 2002. “Péter’s picks & pans: Pans: Wikipedia,” Online, volume 26, number 2, pp. 81–82.

Leonard Kniffel, 2008. “Authority and Wikipedia,” American Libraries, volume 39, number 7, p. 4.

Verne G. Kopytoff, 2011. “BITS; Celebrating 10 years of Wikipedia,” New York Times (17 January), at http://query.nytimes.com/gst/fullpage.html?res=9407E1D6133CF934A25752C0A9679D8B63&scp=2&sq=&st=nyt, accessed 30 January 2011.

Ann M. Lally and Carolyn E. Dunford, 2007. “Using Wikipedia to extent digital collections,” D–Lib Magazine, volume 13, numbers 5–6, at http://www.dlib.org/dlib/may07/lally/05lally.html, accessed 24 October 2010.

Greg Landgraf, 2009. “Wikipedia growth slows,” American Libraries, volume 40, number 11, p. 27.

Sook Lim, 2009. “How and why do college students use Wikipedia?” Journal of the American Society for Information Science and Technology, volume 60, number 11, pp. 2,189–2,202.

Olena Medelyan, David Milne, Catherine Legg, and Ian H. Witten, 2009. “Mining meaning from Wikipedia,” International Journal of Human–Computer Studies, volume 67, number 9, pp. 716–754.http://dx.doi.org/10.1016/j.ijhcs.2009.05.004

Lokman I. Meho and Cassidy R. Sugimoto, 2009. “Assessing the scholarly impact of information studies: A tale of two citation databases — Scopus and Web of Science,” Journal of The American Society for Information Science and Technology, volume 60, number 12, pp. 2,499–2,508.

Finn Årup Nielsen, 2007. “Scientific citations in Wikipedia,” First Monday, volume 12, number 8, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1997/1872, accessed 30 August 2010.

Lauren Pressley and Carolyn J. McCallum, 2008, “Putting the library in Wikipedia,” Online, volume 32, number 5, pp. 39–42.

Lucy Holman Rector, 2008, “Comparison of Wikipedia and other encyclopedias for accuracy, breath, and depth in historical articles,” Reference Services Reviews, volume 36, number 1, pp. 7–22.http://dx.doi.org/10.1108/00907320810851998

Scopus, 2010. Scopus, at http://www.scopus.com, accessed 26 October 2010.

Anselm Spoerri, 2007. “What is popular on Wikipedia and why?” First Monday, volume 12, number 4, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1765/1645, accessed 22 February 2011.

Felix Stalder and Jesse Hirsh, 2002. “Open source intelligence,” First Monday, volume 7, number 6, at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/961/882, accessed 21 July 2011.

Besiki Stvilia, Michael B. Twidale, Linda C. Smith, and Les Gasser, 2008. “Information quality work organization in Wikipedia,” Journal of the American Society for Information Science and Technology, volume 59, number 6, pp. 983–1,001.

Danny P. Wallace and Connie Van Fleet, 2005. “The democratization of information? Wikipedia as a reference source,” Reference & User Services Quarterly, volume 45, number 2, pp. 100–103.

Web of Science, 2010. Web of Science, at http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/, accessed 24 August 2010.

Dennis M. Wilkinson and Bernado A. Huberman, 2007. “Assessing the value of cooperation in Wikipedia,” First Monday, volume 12, number 4, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1763/1643, accessed 20 August 2010.

 


Editorial history

Received 16 March 2011; revised 29 April 2011; accepted 22 July 2011.


Copyright © 2011, First Monday.
Copyright © 2011, Taemin Kim Park.

The visibility of Wikipedia in scholarly publications
by Taemin Kim Park.
First Monday, Volume 16, Number 8 - 1 August 2011
http://firstmonday.org/ojs/index.php/fm/article/viewArticle/3492/3031





A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2014.