Aggregated Trustworthiness Redefining Online Credibility Through Social Validation Authors: Johan Jessen, M.Sc., IT University of Copenhagen Anker Helms Jørgensen, Ph.D., Prof., IT University of Copenhagen Abstract: This article investigates the impact of social dynamics on online credibility. Empirical studies by Pettingill (2006) and Hargittai et al. (2010) suggest that social validation and online trustees play increasingly important roles when evaluating credibility online. This dynamic puts pressure on the dominant theory of online credibility presented by Fogg and Tseng (1999). To remedy this problem we present a new theory we call "Aggregated Trustworthiness" based on social dynamics and online navigational practices. Content: Introduction Theories of Online Credibility Aggregated Trustworthiness Explanatory Power Conclusion Introduction: Credibility of online information has always been a key factor in our understanding of the Web at large. With the recent rise in social media and particularly user-generated content, our need for a solid understanding of online credibility is becoming ever more important (Gillmor, 2008). Recent empirical studies suggest that we put more emphasis on social validation than traditional expert sources when assessing online information (Hargittai et al., 2010; Pettingill, 2006). Yet, our current theories fail to explain this dynamic, leaving us with an insufficient framework to analyze and explain the workings of websites such as Twitter and Wikipedia. The point of departure for this paper is a discrepancy between the leading theory of online credibility and our actual online behavior. We argue that the social dynamics online deeply impact our evaluation process by incorporating other people's evaluation and the navigation process itself. The unique characteristics of online information seeking routines and navigational practices presents a more dynamic theory of online credibility evaluation. We call this theory Aggregated Trustworthiness. Defining Online Credibility Several theories have tried to pinpoint the concepts involved and the workings of online credibility. Fogg and Tseng's paper on 'computer credibility' is broadly cited and generally offers the best theory of online credibility (Fogg & Tseng, 1999). However, their core theory was developed in 1999, and as we will argue, crucial changes in both technological reach and online practices have made their theory of online credibility inadequate for our present day online environment. To understand why it is necessary to explain Fogg & Tseng's theory of computer credibility. They stress that credibility is always a perceived quality and not a property residing in any human or computer product. Rhetoricians use the term ethos to explain how the subjective quality of credibility evaluation (also known as trustworthiness) is primarily established by the receiver of the message, incorporating emotional sentiment and situational context (Bitzer, 1966; Warnick, 2004; boyd, 2008). In other words we cannot design credibility itself, only design for credibility. Fogg uses a simple metaphor to illustrate credibility; like beauty, credibility is in the eye of the beholder and "it exists only when you make an evaluation of a person, object, or piece of information." (Fogg, 2003, p. 122). That said, evaluating credibility is not random. Certain qualities exists to help lay the basis for a website to be considered credible (Fogg et al., 2002). Fogg & Tseng calls this property of credibility for believability[1] (Fogg & Tseng, 1999). In its simplest form credibility is the persuasive nature of the medium (Fogg & Tseng, 1999; Fogg, 2003). Thus, if a webpage is believable it is also considered credible. Fogg & Tseng argue that from the dozen or more elements that contribute to credibility evaluation, there are just two key dimensions of credibility: trustworthiness and expertise (Fogg & Tseng, 1999; Fogg, 2003). Figure 1: Model of the key dimensions of credibility (Fogg, 2003) Fogg defines trustworthiness in the terms well-intentioned and unbiased, and expertise in the terms of perceived knowledge, skill, and experience (Fogg, 2003). According to Fogg & Tseng's theory a high level of credibility incorporates both a high level of trustworthiness and expertise. Hence, according to their theory a website cannot be considered credible if it does not entail both of these concepts. This construct is the basis of their theory on online credibility. Aggregated Trustworthiness: Moving onwards from this theory, online users would tend to look for both trustworthiness and expertise cues to establish a level of credibility of the information in question – effectively mimicking the traditional routine of source evaluation. In this routine a user would look at who wrote the piece of information to assess whether the author is trustworthy and an authority on the matter. If so, strong credentials and unbias would serve as strong credibility cues. In most offline settings this approach is very robust since verifying the identity and credentials of the source are much easier than evaluating the accuracy of their claims. The problem, however, is that a lot of information online is detached from these credential and authority cues. Particularly in user-created content platforms like Wikis, review and rating websites, blogs, forums, Twitter etc. we find very few, if any, direct cues of expertise. Following Fogg and Tseng this dynamic severely lowers the perceived credibility of these types of online information, since we are not able to easily identify the authors and hence evaluate their expertise on the matter. Yet, recent empirical studies suggest that youth do in fact perceive information without identified author as credible (Pettingill, 2006; Hargittai et al., 2010) – especially if the collective judgment of the information is available (Lankes, 2008; O'Byrne, 2009). This means the feedback of others is crucial when assessing the credibility of online information (Weinschenk, 2009; Ljung & Wahlforss, 2008). An extension of this dynamic is the reliance on trustees (Pettingill, 2006). Trustees often act as a form of authority and provides a baseline of trustworthiness. Trustees are not necessarily experts of the specific topic, yet are important fixtures in the evaluation process and the overall dynamic of establishing credibility (Wang & Emurian, 2004). In addition, the navigation process itself impacts the perceived credibility by highlighting search rankings, topic correlations, and brand exposure (Hargittai et al., 2010). The social element attached to information is thus key when users evaluate its credibility. Enabling vote-like behavior such as comments‚ "Likes", ratings, and even links simply provide a much broader spectrum of validation than possible in any offline setting. Collecting multiple streams of trustworthiness cues to form an aggregate of credibility is at the root of this dynamic. We call this theory of online credibility, Aggregated Trustworthiness. The illustration below shows the factors and dynamic of Aggregated Trustworthiness (see Figure 2). Figure 2: Illustration of Aggregated Trustworthiness On the right side of the solid arrow is 'Perceived Credibility' as the degree to which we believe the information presented to us. On the left side of the solid arrow there are three main factors. 'Social Validation' includes the large-scale verifications made by others (e.g. comments, Facebook Likes, shares, social bookmarks, ratings etc.). Social validation may include profiles, but are not constrained to them. In our theory social validation simply means that the more people acknowledge a certain piece of information the more trustworthy it is perceived. 'Profile' provides the baseline for identity online as well as adding a fixture of the evaluation (e.g. LinkedIn profile, Twitter stream, personal website or blog). Having a known identity can be critical when assessing important information. 'Authority & Trustee' includes the known brand or authority on the matter[2] (e.g. New York Times, Stanford University etc.), but also trustees verifying lesser-known sources (e.g. social network 'friends', Wikipedia references, Twitter personas etc.). These three factors are dependent on each other and are thus vetted into a larger system of navigation (the dotted arrows). This dynamic includes basic search and navigational processes (e.g. search context such as history, ranking, lookups, links etc.) Together the model illustrates how social validation may provide verification of an authority, which in turn may provide verification of a specific profile, focusing our evaluation process and establishing the level of perceived credibility of the information. However, we are not suggesting that quantity metrics such as Facebook Likes or Google's PageRank can or should substitute a critical analysis of online information. Yet, we do argue that the theory of Aggregated Trustworthiness explains a dynamic unique to the Web and is made possible by factors not attainable in any offline setting, such as large-scale feedback systems. These elements shift the perception of credibility from necessitating a fixture of traditional expertise cues, to a process which is inherently more dynamic and flexible, not hinging on any root authority role. Changing the dynamics this way effectively spreads out the risk of being mislead from relying on a few stable sources to many, albeit, less stable sources. The key here is how the dynamic functions without cues of root authority, and hence perceived expertise of the author of the information – a dynamic in stark contrast to our current theory of online credibility formulated by Fogg and Tseng (1999). Instead of factoring in the perceived expertise of the sender of the information, we instead leverage the social feedback and collective judgment of the information to assess its credibility. Joining many untrustworthy pieces of information to stitch together a patchwork of credibility is simply easier in an online setting where root authority is much harder to establish (Lankes, 2008; Shirky, 2009). We have based our theory of Aggregated Trustworthiness on two major studies of young adults' evaluation of Web content and information-seeking routines (Hargittai et al., 2010) and youth's online research practices (Pettingill, 2006). Both studies demonstrate how youth gather credibility cues from a broad spectrum of sources, not confined to expert sources. Hargittai et al.'s study even demonstrate how users not exposed to source credentials or traditional expertise cues still manage to successfully complete their given information-seeking tasks (Hargittai et al., 2010). Additionally, a group in one study placed great emphasis on trustees when accessing the quality of information (Pettingill, 2006). A trustee is typically a guiding person they know from an offline setting like a teacher or a parent. Yet, youth involved in social networking had expanded these trustee roles to include members of their online network, and hereby leveraging the flexibility and reach of online social networks when evaluating online credibility. As Pettingill explains: "While subjects were ambivalent about the use of Wikipedia generally, those engaging in social networking sites daily were more likely to cite Wikipedia as a trusted source for information." (Pettingill, 2006, p. 6). The evaluations made by others are thus a key cue to determine the credibility of the information in question (O'Byrne, 2009). Aggregating a wealth of trustworthiness cues provide the most robust form of evaluation, when author credentials are hard or impossible to come by. Understanding the social dynamics online is thus far more important in credibility evaluation than the static methods of traditional source critique. Traditional theories on online credibility, such as the one proposed by Fogg and Tseng, give very little attention to this fact. Root authority and source critique are still important factors (e.g. when analyzing author intent), but it is not the preferred method of evaluating online credibility. Using our theory we can explain why a number of Web platforms from Wikipedia to Twitter are perceived as credible, despite the lack of known authorities and traditional expertise cues. Looking to other people's evaluation (e.g. Likes, shares, comments etc.) or the aggregate of this behavior (e.g. search ranking, Twitter trending topics etc.) provides a baseline of judgement and helps our own evaluation (Shirky, 2009; Weinschenk, 2009). Key in this dynamic is how users levitate the built-in filtering mechanism of the Web such as search and social recommendations, putting greater emphasis on the navigational tools and processes than the expertise of the source (Lankes, 2008; Shirky, 2008; Hargittai et al., 2010). The core values are the same, but the process is different. In the following we will illustrate our theory's explanatory power by using Wikipedia as a lead example. Explanatory Power: The Aggregated Trustworthiness theory helps explain the success of a number of online platforms like Wikipedia, eBay, Twitter, LinkedIn and many more. Wikipedia has proven exceptionally difficult to explain by traditional theories of credibility (Warnick, 2004), and hence it offers a prime example to explain and highlight the explanatory power of our theory. The platform of Wikipedia have brought with it an array of obstacles in terms of evaluating credibility, verifiability, and consensus of interpretation. Focusing only on credibility of edits we are faced with the following scenario. Normally, in an offline setting, the author of a text is known and the time of publication is fixed. But not on Wikipedia. What makes Wikipedia possible in the first place, i.e. the networked structure and post-hoc review process, is also what makes it practically impossible to critically evaluate using the traditional methods of source evaluation [3] (Standler, 2004; Stvilia et al, 2005). Evaluating a Wikipedia article by looking for known author or time of publication is essentially meaningless due to its format and the dynamic nature of the medium (Warnick, 2004). Since articles are written and rewritten by multiple contributors, very few of which may be said to be a known authority on the matter (or visible to anyone outside the Wikipedia community), the traditional approach of evaluating author credentials and point-of-origin simply breaks down. The collaborative nature of Wikipedia is thus limiting for the traditional method of credibility assessment, but actually advantageous for the aggregated kind. The vast majority of the editors and contributors on Wikipedia are anonymous. Yet combined and added together, the sum of their edits and re-edits seems to justify the inherent lack of root authority and identity. Seemingly, we do not trust the individual anonymous user, but do we trust a lot of them (Lankes, 2008). One explanation lies in the fact that it is extremely cumbersome to individually coordinate or influence the actions of a very large number of people (Shirky, 2008). Especially, when all their actions are transparent to one another. It is simply unlikely that a contributor on Wikipedia had bribed all the other editors; it is much more likely that they independently from each other found the original contribution acceptable. According to Shirky it is the incredible amount of social media that necessitates this post-hoc filtering approach: "The expansion of social media means the only working system is publish-then-filter." (Shirky, 2008, p. 98). The dynamic of Wikipedia's post-hoc filtering system is dependent on social cues, broadly defined as our reactions to certain behavior characteristics, social validation principles, and general reinforcements through other people's judgements (Resnick et al., 2006; Weinschenk, 2009). By consulting what (a lot of) other people think we are presented by a form of transparency normally unavailable – be it news articles, forum posts, or blog comments. The opinions of others matter to us and studies done on whether online recommendations influence buying decisions showed a volume increase of 20 percent for items with recommendations over items without them (De Vries & Pruyn, 2007). Since we react favorably to social validation, other people's collective judgement is perceived as a trustworthiness metric. We even seem to react to artificial mimicry of social cues as a way to evaluate trustworthiness cues directly from computers (Mui, 2002; Nass & Reeves, 2003). This tendency of social validation and reliance of social cues is a key factor in the dynamic of Wiki-edits and a founding principle in our theory of Aggregated Trustworthiness. A research team from the University of California, Santa Cruz has even taken this dynamic a step further and developed an algorithm called WikiTrust [4]. Critics of Wikipedia have argued that there is no apparent way to see which articles, and more importantly what part of the article, is credible and which is not (Stvilia et al., 2005). To counter this critique and offer a solution using the process of multiple edits, social validation principles, and post-hoc filtering, the research team has developed WikiTrust. The algorithm works by color coding every word in every article based on the reliability of its author and the length of time the edit has persisted on the page. The algorithm counts the number of silent edits any word has undergone and assigns a value to the author. The more unedited pieces of text an author has on Wikipedia, the higher the reputation [5] within the algorithm the author will get. Newly added edits from questionable sources are highlighted in bright orange, and the longer it stays on the page, literally surviving multiple edits, the lighter shade of orange it will get until it is white (see Figure 3). Figure 3: An example from WikiTrust: The Wikipedia page "Politics of Denmark". Notice the error "Anders Fjogh Rasmussen" highlighted in bright orange. The word Fjogh is a play on the prime minister's real middle name Fogh and literally means fool in Danish. Screen capture: http://wikitrust.collaborativetrust.com/screenshots (Accessed 29-07-2010) Wikipedia and particularly the WikiTrust algorithm are clear examples of the explanatory power of the theory of Aggregated Trustworthiness. Embedding the social verification into the edit and navigation process itself, we see how users are able to circumvent a credibility evaluation based on expertise. WikiTrust is just one example of a service letting people assess the credibility of online information based on the collective judgments made by other people. Conclusion: Aggregated Trustworthiness provides a more adequate theory of online credibility. Incorporating social validation, online trustees, and profile-based websites, our theory is a first step to better explain the processes of credibility evaluation of online information and platforms lacking traditional expert cues. Highlighting recent empirical studies we have argued that information without explicit cues of authorship and expertise are in fact, and contrary to our present theories, perceived as credible. To illustrate the explanatory power of our theory we have focused on the social dynamics of Wikipedia and how these dynamics mitigate the need for expertise to establish credibility. Our theory of Aggregated Trustworthiness is based on this social dynamic and points in a new direction for future research of online credibility. About the Authors: Johan Jessen (M.Sc., IT University of Copenhagen) has studied online trust and credibility at the Department of Communication at Stanford University in 2009-2010 and is attributing author to Howard Rheingold's book on digital literacy (in press). Direct comments to juje [at] itu [dot] com Anker Helms Jørgensen (Ph.D., University of Copenhagen) is Associate Professor at the IT University of Copenhagen with a primary research interest in Human-Computer Interaction. Notes: 1: A point worth noting is how Fogg and Tseng argue that the concepts of 'trust' and 'credibility' are not the same, and have unfortunately been used imprecisely and inconsistently in much of the relevant academic literature (Fogg & Tseng, 1999). They stress that 'trust' should be viewed as dependability and 'credibility' should be viewed as believability (Fogg & Tseng, 1999; Fogg, 2003). Related, yet not interchangeable. For the sake of brevity and clarity we will not challenge their definitions. Throughout the article we will refer to credibility as the degree to which we believe the information presented to us. 2: An interesting point of view is how authority and trustees do not have to be human. In a speculative blog post Shirky argues that computer algorithms are increasingly exhibiting the same traits as a human authority and, more importantly, also increasingly being perceived as authority (Shirky, 2009). A point which echoes much of the academic literature on the subject (Mui, 2002; Nass & Reeves, 2003). 3: Nature did a study in 2005 showing that Wikipedia was just shy of being as accurate as Encyclopedia Britannica (EB). Based on 42 articles Nature found 123 factual errors in EB and 162 in Wikipedia. EB refuted the findings, among other things claiming greater severity of errors on Wikipedia, but Nature held on to their original findings. See: http://www.nature.com/nature/journal/v438/n7070/full/438900a.html (Accessed: 22-02-2010) 4: http://wikitrust.soe.ucsc.edu/ (Accessed 27-06-2010) 5: An important point to note about the WikiTrust algorithm is how it does not measure the quality of the edits or the actual claims within the text. WikiTrust only measures credibility derived from consensus. If the majority of editors on a given topic does not correct an erroneous statement, it stays and thus builds up its 'credibility score'. This is not just the weak point of WikiTrust but in essence all systems incorporating an editorial ‚- user-generated or not. References: Lloyd F. Bitzer, 1966. "The Rhetorical Situation", Philosophy and Rhetorics Vol 25, 1992, p. 1-14. Online: http://www.jstor.org/pss/40237697, Accessed: April 2010 danah body, 2008. Taken Out of Context. American Teen Sociality in Networked Publics, Dissertation submitted at the Department of Information Management and Systems, MIT. Online: http://www.danah.org/papers/TakenOutOfContext.pdf, Accessed: 27 April 2010 Peter De Vries and Ad Pruyn, 2007. "Source Salience and the Persuasiveness of Peer Recommendations: The Mediating Role of Social Trust". In Persuasive Technology, Second International Conference on Persuasive Technology. New York: Springer. B.J. Fogg, 2003. Persuasive Technology: Using Computers to Change What We Think and Do, Morgan Kaufman, USA B.J. Fogg, 2002. Cathy Soohoo, David Danielson, Leslie Marable, Julianne Stanford, Ellen R. Tauber, Report: How Do People Evaluate a Web Site's Credibility? Results from a Large Study. Stanford, CA, USA. Online: http://www.consumerwebwatch.org/pdfs/stanfordPTL.pdf, Accessed: 15 January 2010 B.J. Fogg and Hsiang Tseng, 1999. "The Elements of Computer Credibility", In CHI '99 Pittsburgh, PA, USA, ACM 1999 Eszter Hargittai, Lindsay Fullerton, Ericka Menchen-Trevino, Kristin Yates Thomas, 2010. "Trust Online: Young Adults' Evaluation of Web Content" in International Journal of Communication 4 (2010), pp. 468-494 Dan Gillmor, 2008. "Principles for New Media Literacy" in Re:public, Berkman Center for Internet and Society, Harvard University. Online: http://cyber.law.harvard.edu/sites/ cyber.law.harvard.edu/files/Principles%20for%20a%20New%20Media%20Literacy_MR.pdf, Accessed: 27 April 2010 R. David Lankes, 2008. "Trusting the Internet: New Approaches to Credibility Tools." Digital Media, Youth, and Credibility. Edited by Miriam J. Metzger and Andrew J. Flanagin. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press, pp. 101-122. Alexander Ljung and Eric Wahlforss, 2008. People, Profiles & Trust. On Interpersonal Trust in Web-mediated Social Spaces. Berlin, Germany, ISBN: 978-1-4092-2942-1 Lik Mui, 2002. Computational Models of Trust and Reputation: Agents, Evolutionary Games, and Social Networks. Dissertation submitted at Department of Electrical Engineering and Computer Science, MIT, 2002. Online: http://groups.csail.mit.edu/medg/people/lmui/docs/phddissertation.pdf, Accessed: 15 January 2010 Cliff Nass and Byron Reeves, 2003. The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places, Cambridge Press, USA William Ian O'Byrne, 2009. Facilitating Critical Thinking Skills through Content Creation: Results from a Pilot Study, Neag School of Education, University of Connecticut. Online: http://www.scribd.com/doc/47085454/Facilitating-Critical-Thinking-Skills-Through-Content-Creation, Accessed: 25 June 2011 Lindsay Pettingill, 2006. "Trust Without Knowledge: How Young Persons Carry out Research on the Internet", in GoodWork Project Report Series, Number 48, Harvard University, USA Paul Resnick, Richard Zeckhauser, John Swanson, Kate Lockwood, 2006. The Value of Reputation on eBay: A Controlled Experiment. Online: http://www.si.umich.edu/~presnick/papers/postcards/, Accessed: 27 April 2010 Clay Shirky, 2008. Here comes everybody, Penguin Press, USA Clay Shirky, 2009. A Speculative Post on the Idea of Algorithmic Authority. Online: http://www.shirky.com/weblog/2009/11/a-speculative-post-on-the-idea-of-algorithmic- authority/, Accessed: 15 January 2010 Ronald B. Standler, 2004. Evaluating Credibility of Information on the Internet. Online: http://www.rbs0.com/credible.pdf, Accessed: 27 April 2010 Besiki Stvilia, Michael B. Twidale, Les Gasser, Linda C. Smith, 2005. "Information Quality Discussions in Wikipedia", presented at ICKM05. Online: http://mailer.fsu.edu/~bstvilia/papers/qualWiki.pdf, Accessed: 25 June 2011 Ye Diana Wang and Henry H. Emurian, 2004. "An overview of online trust: Concepts, elements, and implications", In Computers in Human Behavior 21 (2005) 105-125, Elsevier Barbara Warnick, 2004. "Online Ethos - Source Credibility in an "Authorless Environment"", in American Behavioral Scientist, vol 48. No. 2, (pp. 256-265) Susan M. Weinschenk, 2009. Neuro Web Design: What Makes Them Click?, New Riders Press, USA License: Creative Commons License
Aggregated Trustworthiness, Redefining Online Credibility Through Social Validation by Johan Jessen & Anker Helms Jørgensen is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License.