This study is the first large–scale multi–method attempt to empirically examine the characteristics leading to development of expertise in EverQuest II, a popular massively multi–player online role–playing game (MMOs). Benefiting from the unprecedented opportunity of obtaining game log data matched with survey data, the project investigated the relationship between player motivations and in–game behavior, personality characteristics, and demographic attributes with game performance and achievement, which we refer to as game “expertise.” Players who were high on achievement motivation or social motivation had higher game expertise, while those high on immersion motivation had lower expertise. Game experts were also characterized by focusing their game time on completing tasks. Younger players showed a slim advantage over older players. Male and female players exhibited similar expertise levels in this MMO.
In recent years, video games have been the subject of increased interest by researchers in a variety of disciplines including communication , education , and economics . This interest is in part due to the increased complexity and realism captured by contemporary video games. These games — such as EverQuest and World of Warcraft — not only generate millions of dollars in revenue for their respective developers, but also support a robust in–game economy of currency , and act as platforms for social communities , . As a result, they offer researchers an opportunity to study both the contributors to and consequences of in–game interactions. Over the past several years, advances in computer processing power, along with increases in the speed and capacity of digital networks have contributed to the rise of games that can be characterized as persistent virtual worlds — online environments that remain active despite any single player logging off. One subset of these games is Massively Multi–player Online Role–Playing Games (MMOs) that allow participants to control characters that navigate these virtual worlds and complete a series of tasks. MMOs, like EverQuest, present a fundamentally different research setting from other persistent virtual worlds such as Second Life because at their core they are designed to be task–oriented and as a result create an environment of relative achievement among participants.
The popularity of virtual worlds, as well as the tangible rewards associated with success invites further examination of in–game play. Researchers have argued that the complex, immersive, and socially interdependent nature of persistent online worlds make them sufficiently similar to “real life” that we can gain insights into social, behavioral, and economic phenomena by studying games as a proxy for off–line arenas , , . The advantage of studying analogs of real life in game settings is that researchers have access to data that they would not have in real life, such as exact measures of, achievement, rewards, and behavioral records. Moreover, game logs enable researchers to follow changes in these measures over time at a level of detail impossible in real life settings. With the availability of rich data such as game logs, it is possible to go beyond the research on the general effects of game play and to study game behaviors in detail. This study will focus on the expert players in EverQuest II and the predictors of their expertise. Specifically, to what extent are characteristics such as motivations to play, game focus, personality, and demographic attributes, associated with higher game expertise?
Expertise has often been understood in terms of superior performance . Some scholars argue that superior expertise is attributable to innate traits or special abilities, while others suggest that expertise is acquired through experience, practice and learning , , . However in most domains, information on the objective measures of expertise and performance is hard to collect. Universal standards for expertise and performance may not exist, let alone detailed records of experts’ and non–experts’ behaviors.
This study attempts to overcome these limitations by investigating expertise in the virtual world, or in MMOs, to be more specific. Within each game, there are universal objective measures for any player’s expertise and achievement. Game logs provide detailed records for the developmental history of expertise and achievement and the records for the player’s in–game behaviors. In addition, survey data, including but not limited to game players’ demographic information, personalities, and game play motivations, are matched to behavioral data in the game logs, which provides a more complete picture of the game expertise and experts.
In addition to his own empirical work in the field of psychology, Ericsson has provided an insightful review  on the study of expertise and experts. In the sixteenth century, eminent artists and scientists were believed to have divine gifts, which make them qualitatively different from others. Galton  associated the existence of superior expertise to inherited differences in human brains and nervous systems. Alternatively, De Groot  shifted the focus from physically inheritable characteristics to thought processes, after scientists failed to find significant differences between experts and non–experts in terms of general mental speed, basic memory, and intelligence with psychometric tests. Simon and Chase  proposed the first general theory of expertise, and found that chess masters’ expertise primarily derived from their prior chess knowledge and extensive chess experience. Studies on a variety of domains further supported their findings that expertise can be obtained by learning and practice.
Different definitions and assessment criteria have been proposed for expertise . In this study, we take the relative approach , in which expertise is considered a continuous rather than binary attribute. It is indicated through achievement relative to peers in similar conditions. It can be accumulated, and indicates a level of proficiency that novices can achieve with training . Consequently, we define experts as those who are more knowledgeable, skillful and have higher levels of performance relative to others attempting like tasks.
Expertise research has attracted attention from researchers in various fields, such as psychology, sociology, computer science, education, communication, and management. Different strands of research hold different goals, two of which particularly interest the authors. One is expert identification in order to extract relevant knowledge. The other is to improve the training of non–experts to make them more knowledgeable and skillful. The first goal focuses on obtaining a collection of reliable attributes that facilitate quick and accurate expert identification. Demographic data, the most easily available information, are frequently used for population segmentation and identification. Therefore, in this study we first examine the relation between players’ demographic characteristics and their game expertise.
Much of the existing MMO research has aimed to determine the type of individuals engaged in the games , , . These studies have helped debunk many of the stereotypes regarding video game players, while confirming others. Online games are not dominated by adolescent players, as previously thought. Instead, multiple studies of MMO play indicate that adults above the age of 25 make up the majority of participants , . For EverQuest II, there are more players in their 30s than in their 20s, and older players tend to play more hours per week than younger players . As the empirical findings on the player population are so different from the traditional image, it calls into question the traditional stereotype of the power players, or the game experts. Who have higher game expertise, the younger players or the older players?
In looking at gender, previous work confirms that males tend to dominate MMO play, with Griffiths, et al.  and Williams, et al.  finding that around 80 percent of game players are male. In Lucas and Sherry’s study , female participants played video games less frequently. One explanation for this gender difference is that most computer games are designed with males in mind , . Findings in a survey of video game players by Sherr, et al.  supported this explanation. They found that women are more likely to engage in game situations that involve socializing, and are less likely to enjoy games played for competition or involving three–dimensional rotation. Similarly, Lucas and Sherry  also found that female participants are less interested in game genres featuring competition and three–dimensional rotation. As most computer games are about competition, and socializing has just become a theme in the games very recently, it is reasonable to expect the experts, i.e., the power players, are mostly male. However, Williams  found that while male players constitute the majority in EverQuest II, female players actually play slightly more hours per week than male players. Also, the big difference between the male and female ratio in the U.S. population (49.1 percent vs. 50.9 percent) and in MMOs population (80 percent vs. 20 percent) may indicate that women playing MMOs are a particular subset of the gender that are attracted to the game or possess higher game aptitude . The theoretical and empirical research reviewed above suggests that, which groups, in terms of age and gender, tend to have higher game expertise, remains an open question. Therefore, we propose the following research question:
RQ1: What is the relationship between age, gender and game expertise?
Motivations, game behaviors, and personalities
A second goal that motivated this study is to find out what attributes of game players may contribute to expertise development. Therefore, this study aims to provide some understanding on how to improve the training process for non–experts by detecting some player attributes that can be cultivated or need to be constrained.
RQ2: What other characteristics are associated with higher game expertise?
Motivations. Many scholars have emphasized the importance of motivations and interests in expertise formation. Ackerman  found that knowledge bases, which are critical components of expertise, correspond to their interests. Ericsson’s review  pointed out that among the conditions for optimal learning and improvement of performance, the most cited condition is motivation to attend to the task and exert effort to improve their performance. Among Sternberg’s  five key elements in the model of developing expertise, motivation is considered most important.
The richness of features, social environment and diversity of game play in MMOs means that individuals with various motivations can sustain interest in the game. The relative expertise of players at any given point can be characterized as a result of their choices in how to navigate the game world in a way that is of greatest interest to them. Early qualitative research on MMOs by Bartle  categorized players into four distinctly motivated groups — Achievers, Socializers, Explorers, and Killers. Subsequent work by Yee  used data from a survey of game users and found three second–order motivation categories — Achievement, Immersion, and Social. Game players who are high on achievement motivation consider it most important to level up, acquire great items and gear, and compete with other players. Players considering immersion as the primary motivation focus on exploring the game world, building unique characters, and role playing. Players holding social motivation highly value the opportunity and experience to work in teams and develop deep and meaningful relationships with other players.
Each of these motivations has been put forth as a primary driver of game enjoyment. Different motivations drive different activities, to which the players dedicate their time and efforts, and give rise to different game outcomes including expertise development. Vorderer, et al.  argue that individuals playing interactive games are attracted by the social competition of the environment, which is associated with game achievement. Therefore, the players with achievement motivation are expected to have high levels of game expertise, because “they are strivers, constantly trying to better themselves and their accomplishments.”  In order to gain high levels, obtain valuable and rare treasure or gear, and be more powerful than other players, individuals need to improve their game knowledge and skills with constant learning and repeated practice.
Players with high social motivation treat the game as primarily another environment to meet other people, develop social relations, and work in teams. While working in teams may help improve game–specific knowledge or skills, meeting people and establishing social relations are not necessarily related to the development of game expertise. In fact, if the players spend most of their time socializing, it is reasonable to expect that social motivation will be negatively related to game expertise.
The players with immersion motivation enjoy exploring the game world and learning what others do not know, creating unique characters, and role playing. They also tend to consider the game as an escape from the real world. It is important to note that the concept of immersion is different from the concept of engagement. The typical activities of immersion include role playing, building unique characters, escaping from the real world, and exploring the game world. Except for the last element, these activities focus on individualistic immersion rather than engagement in game tasks or interaction with other gamers. Compared to those with other motivations, these players seem more satisfied with enjoying the small world they create for themselves rather than obtaining achievement or establishing social relations in the “outside” world. Therefore, the first set of hypotheses is:
H1a: Players who are high on the achievement motivation in game playing will have higher levels of expertise.
H1b: Players who are high on the social motivation will have lower levels of expertise.
H1c: Players who are high on the immersion motivation will have lower levels of expertise.
Game activity: focus. A consensus among expertise researchers is that “experts work at becoming experts” . Thorndike  found that even for tasks that are frequently conducted, adults perform at a level much lower than their maximal level. In order to achieve high expertise and outperform others, experts need to attend to their tasks and focus their activities on learning and practicing in the specific domain. Therefore, for game players, all time spent logged into the game does not contribute to expertise development equally. As MMOs provide a wide range of potential activities in their virtual worlds, players are likely to exhibit various distributions of their game time. For example, some may spend most of their time completing quests and killing monsters, while others chatting with their friends. Therefore, becoming an expert is not merely a result of playing the game, but is a result of a specific type of focused game play. Therefore, we predict:
H2: Players who focus their game time on completing quests will have higher levels of expertise.
Personality traits. In addition to game play motivations and game focus, players’ personality traits can also affect how they accumulate expertise in the game environment. Many tasks in MMOs are very complicated. To be a game expert, players need to obtain a large amount of game–specific knowledge and master many skills. For some of these knowledge and skills, the players may be able to learn by themselves, but learning from other players can be a more effective and efficient alternative. There are also many skills that are required for complicated tasks and can only be learned and practiced in group interaction. As MMO environments are by nature highly social , ,  and interaction between players is a central element of these virtual worlds, being extroverted and more sociable can facilitate the players’ learning and practice. In examining the relationship of roughly 2,000 MMO players, Seay, et al.  found that individuals who reported low group activity off–line also reported low group activity in games. Therefore, their off–line personalities are likely to affect their in–game behaviors, which further influence their expertise development. Therefore, we expect:
H3a: Players who are more extroverted will have higher levels of expertise.
There is a long history of interest in the relationship between aggression and gaming . This effects–based field is largely interested in whether aggressive behavior is a consequence of video game activity. Specifically looking at MMOs, Williams and Skoric  found that preferences for graphic violence did not interact with game play to result in increased aggression. This study examines the relation between aggression and game play in the opposite direction: Whether aggressive personality aids improvement of online game expertise. Kim  noted that the “hard–core gamer,” prefers violent, action–intensive games. Given their combative nature, most MMOs fall into this category. Therefore, a tendency toward aggression may help improve game expertise and performance.
RQ3. Are players who self–report more physical aggression also have higher levels of expertise in the game?
Introduction to EverQuest II
The current study focused on EverQuest II (EQII), which is a fantasy–based MMO that allows players to create characters from one of 16 races and explore virtual lands while completing quests, acquiring treasure and advancing their abilities and skills. For example, a player might choose to create the character of a female human priest, acquiring new healing powers and other talents as she gains experience from completing tasks and defeating monsters. These experience points increase her character level, which has a maximum level of 70. Along with special offensive and defensive abilities, characters continue to improve through the use of new armor, weapons and other items. These items can be purchased from vendors, crafted by other players or acquired by adventuring through dungeons or special quests. Characters can also choose to develop various trade skills such as gathering herbs, mining ore, creating potions, tailoring or enchanting items.
Players often join other players and work together to complete more difficult quests or kill more powerful monsters. Some game content actually requires a minimum of five or ten players to band together to even enter the terrain. These teams are often formed to capitalize upon the resources of the various characters. For example, a five–person team might use a fighter to attack first and absorb the majority of the damage, while various other classes deal damage to the monster, and a healer sits back and keeps the team alive. When players are not questing, they can chat with others using the in–game chat system or trade items such as potions or enchants. At any given time, thousands of players can be congregated at a major city where they interact, trade or mobilize.
Sample and procedure
An online Web–based survey was conducted among the EQII players, held by Sony Online Entertainment, the game producer. After logging into the game, the players were invited to participate in the survey, and in turn they were promised a special virtual item, “Greatstaff of the Sun Serpent,” as compensation. After the participants provided informed consent, they proceeded to complete the survey, which took 25 minutes on the average. The cap for the survey sample was set at 7,000, and was reached within three days.
The game log data were collected from four servers in EQII for three months, which consisted of 30,745 players. It contained thorough records on the characters’ status, development, and in–game behaviors. The researchers received the survey data and log data after Sony Online Entertainment removed any identifying information from the dataset. For this study, we extracted data focused on character information, including their attribute statistics, amount of play time, and the total number of completed quests. Finally, the survey and log data were combined, which created a dataset with information both on the players’ off–line attributes, such as demographics and personalities, and on their in–game attributes and behaviors.
One issue arose when matching the survey data and log data. Many players have multiple characters in the game. On the average, each player in the sample had five characters, and some players had up to 26 characters. Since this study focuses on what player characteristics are associated with high game expertise, the player was used as the unit of analysis, and only the primary characters for the participants were included. The primary character was defined as the one the player spent most time playing. This resulted in the final sample of 6,122 players with their primary characters. Out of the 6,122 players, 4,920 (80.4 percent) were male, and 1,202 (19.6 percent) were female. The average age was 31.47 years old with a standard deviation of 9.73 years.
Expertise. As the dependent variable, game expertise was measured as the sum of each character’s statistics on strength, agility, intellect, wisdom, and stamina, which serve as the primary character statistics in EQII. In EQII, players advance in the game through the completion of directed tasks (or “quests”). Upon successful completion of the increasingly difficult quests, players earn achievement points and virtual gear to help them in game play. Players also advance through proscribed status levels, from a new player at level zero, to an expert player who has achieved the maximum level of 70. Figure 1 shows the player sheet with all possible player statistics.
Figure 1: Example of character sheet highlighting the character statistics and the bonuses from particular gear or treasure.
As characters gain levels and obtain gears, their primary statistics improve, which makes them more powerful. Some statistics are more important for certain character types (e.g., priests favor wisdom, warriors favor strength), but the sum of all statistics provides an index that indicates the overall game expertise and applies to all players.
We focused on character statistics as a way to further differentiate players who had already reached the maximum level available in the game. For all characters at level 70, those who have acquired more valuable and rare pieces of gear will have a higher sum of character statistics. A special shield, for instance, can add an additional 10 points on Stamina. We argue that players who get the best gear are further differentiated from the level 70 cap because these items can only be acquired by completing the most difficult game content, including raid encounters that require dozens of people to collaborate in order to defeat an adversary. This high echelon of gear provides players with slightly better statistics than their level 70 peers. Therefore, we calculate expertise by taking the sum of the primary statistics for strength, agility, wisdom, intelligence and stamina.
Player motivations. The measures for player motivations were adapted from those in Yee , with one additional item on chatting, which yielded 11 items in total. A principal component analysis (PCA) with varimax rotation revealed three major player motivations, which were in agreement with the findings by Yee . The achievement motivation focuses on advancement in the game, understanding game mechanics, and competition with other players. The social motivation focuses on socializing, building long–term meaningful relationships, and participation in team work. The immersion motivation focuses on exploration in the game world, role playing, customization of characters, and escapism. The average score for each major motivation was calculated and included as a predictor variable in the final model.
Game focus. Game focus represents the amount of time that players spend doing quests as opposed to other game activities such as utilizing trade skills or socializing with other players. The variable was calculated by dividing the total number of quests that players completed by the log-transformed play time.
Aggression. This study focused on physical aggression because it is more likely to manifest itself in the actions in the games. The measure of physical aggression was from the AQ Physical Aggression subscale in the Aggression Questionnaire . On nine statements, survey participants were asked to indicate the degree of their agreement on a seven–point scale ranging from “Strongly Disagree” (1) to “Strongly Agree” (7). An illustrative statement involving physical aggression was “If somebody hits me, I hit back.” The average score across the nine items was calculated as the players’ score on physical aggression.
Extroversion. The measure of extroversion was partial adaptation from Bendig . Survey respondents were asked to judge how much they agreed with 10 statements on a seven–point scale ranging from “Strongly Disagree” (1) to “Strongly Agree” (7). One example item was “I like to mix socially with people.” The average score across the 10 items was calculated as the players’ score on extroversion.
The actual questions used to measure motivations, aggression and extroversion are included in the Appendix.
Age and gender. On the survey, players were asked for basic demographic information such as their age and gender. The age range was coded from 12 to 65, with “12” indicating “under 13” and 65 indicating “65 and above.”
Play time. Play time was included in the model as a control variable, because we define experts as those who make achievement faster rather than just those who achieve more over time. In most MMOs, players make achievement in large part by simply playing more. However, for our definition of experts, those with higher performance, they should not only make high achievement, but also make it fast. Therefore, what really matters is the rate of accumulation.
Table 1 provides some descriptive statistics for the variables of interest. An alpha level of .01 was used for all statistical tests. The first research question asked whether gender and age are associated with higher game expertise. A correlation (see Table 2) was run between all predictor variables and game expertise. Except for players’ age (r = -.02, p = .23) and gender (r = -.01, p = .36), all other variables were significantly related to expertise level.
Table 1: Descriptive statistics for the variables of included in the model. Note: a. For age, 12 includes all ages below 13, and 65 includes 65 years old and above; b. The variable Time playing has been log–transformed (base–10). Variable N Mean SD Min Max Expertise 6122 122.09 66.16 14.60 315.80 Agea 6073 31.47 9.73 12.00 65.00 Achievement 6008 3.45 .89 1.00 5.00 Motivation Immersion motivation 5980 3.29 .91 1.00 5.00 Socializing motivation 5966 3.34 .73 1.00 5.00 Time playingb 6122 6.39 .58 2.49 7.60 Game focus 6122 55.77 39.32 0 295.52 Extroversion 5413 4.15 1.01 1.00 7.00 Agression 5264 3.22 1.08 1.00 7.00
Table 2: Correlation matrix for variables of interest. Note: N = 6,114; Bivariate correlations, two–tailed tests; a. This variable was log–transformed; * p<.05, ** p<01. Variable 1 2 3 4 5 6 7 8 9 1. Expertise — 2. Gender -.01 — 3. Age -.02 .11** — 4. Achievement Motivation .15** -.16** -.16** — 5. Immersion Motivation .11** .01 -.11** .31** — 6. Socializing Motivation -.03** .03* -.10** .20** .46** — 7. Time playinga .71** .05** .12** .07** .08** -.02 — 8. Game focus .63** .07** .14** -.04** .05** -.01 .64** — 9. Extroversion .01 -.03 -.10** .04** -.021 -.01 -.02 -.05** — 10. Agression .04** -.24** -.22** .13** -.03 .04* -0.03 -.09** -0.02
Next, a linear multiple regression model (see Table 3) was used to examine the relationship between the predictor variables and a player’s expertise level. The model was significant (R = .76, adjusted R2 = .58, F(9, 4896) = 764.95, p < .001). The effect of achievement motivation (B = 7.66, SE = .75, β = .10, p < .001) and social motivation (B = 3.00, SE = 1.00, β = .03, p = .002) were positive on game expertise. Hypotheses 1a was supported, while hypothesis 1b was not supported and the result was in the opposite direction. The effect of immersion motivation (B = -5.63, SE = .76, β = -.08, p < .001) was negative. Hypothesis 1c was supported. The effect of game focus (B = .58, SE = .02, β = .34, p < .001) was positive. Hypothesis 2 was supported.
Both extroversion (B = 1.85, SE = .62, β = .03, p < .01) and physical aggression (B = 3.03., SE = .60, β = .05, p < .01) had significant and positive effects. Hypotheses 3a and 3b were supported. For the demographic variables, age had a significantly negative effect (B = -.01, SE = .001, β = -.10, p < .001), while gender (B = -.03, SE = .03, β = -.01, p = .20) continued to have no significant effect on game expertise. Play time (B = 55.35, SE = 1.39, β = .49, p < .001) as the control variable had a positive and significant effect.
As game focus was calculated as the total number of quests completed divided by play time, there is a concern of potential collinearity between game focus and play time. As shown in Table 2, the correlation between play time and game focus was .64. A collinearity test was also conducted. All the tolerance statistics were above 0.5, and all VIFs were below 2. No collinearity problem was detected.
Table 3: Results of multiple regression analysis for variables predicting player expertise . Note: * p<.05, ** p<.01, *** p<.001. Variable B SE β Intercept -276.50*** 9.20 Achievement motivation 7.66*** .75 .10 Immersion motivation -5.63*** .76 -.08 Social motivation 3.00** 1.00 .03 Game focus .58*** .02 .34 Time playing 55.35*** 1.39 .49 Physical aggression 3.03*** .60 .05 Extroversion 1.85** .62 .03 Age -.71*** .07 -.10 Adjusted R2 .58 F 764.953***
The purpose of this study was to examine the relationship between player motivations, personality characteristics, game focus, and demographic attributes with game expertise. The results from the correlation test were a bit surprising in that neither age nor gender was related to game expertise. In the regression model, after controlling for the effects of all other variables, gender was still not a significant predictor for game expertise, though age showed a negative effect. Therefore, male and female players exhibit similar levels of game expertise in EQII, while younger players tend to have a slight advantage over the elder ones.
The lack of gender differences in game expertise further challenges the stereotypical image of computer game players. One potential explanation is that with the wide range of potential activities available in EQ2, female players have found what they enjoy and are good at it. Another explanation is that the female players in EQ2 are a subgroup that is quite different from the general female population. With the big difference between the male and female ratio in the U.S. population (49.1 percent vs. 50.9 percent) and in the EQ2 population (80 percent vs. 20 percent), the effect of self–selection cannot be ignored. This self–selection bias was found in a study by Williams , which found that female MMO players are much more likely to report being bisexual compared to the general female population. Though these two explanations have different indications, one common implication is that while EQ2 female players are still the minority, they are rapidly becoming an important sub–population.
As expected, players high on achievement motivation tend to have higher game expertise. This is in agreement with existing literature on expertise research that a strong interest in obtaining high performance can lead people through effortful learning and practice, which are necessary for expertise improvement. Also as expected, players high on immersion motivation tend to have lower game expertise. This is because the typical game activities driven by immersion motivation are not related to learning and practicing how to complete game tasks. Since our expertise measure is based on achievement, these activities will not contribute to expertise improvement.
Contrary to our prediction, social motivation was found to be positively associated with higher expertise. Socializing does not distract players’ attention from improving their expertise and performance. By contrast, it helps them develop higher game expertise. One explanation is that MMOs are distinctively different from previous game genres. Some researchers have pointed out that MMO environments are by nature highly social , , . There are many social structures in the game, and interaction between players is a critical element in the virtual world. It requires interaction with other players in order to learn game knowledge and practice many complicated skills. Being interested in working in groups facilitates the learning and practice processes. Some social network research found that people benefit most from the experts with whom they are familiar . Therefore, pro–social gamers are likely to have more social ties, and pre–existing social ties provide them with access to the knowledge and skills held by other players. In addition, the interest in meeting people may also help the players find the appropriate people to form a superior group, which enables them to obtain higher expertise and achievement.
The results suggest that task focus contributes to game expertise. This shows that concentrated experience in the domain is important for expertise development. In short, individual time arrangement affects game outcome. Players who spend more time working on tasks obtain higher game expertise.
Finally, in terms of personality characteristics, players who score higher on the physical aggression scale tend to obtain higher game expertise. However, because this variable is self–reported, it is difficult to assess whether these gamers are actually physically aggressive. Therefore, this finding should be taken with caution.
Second, players who are more extroverted also tend to have higher expertise. Being more sociable facilitates learning and expertise accumulation. In hindsight, this finding is consistent with our unexpected result indicating that social motivation leads to higher expertise.
Play time was found to positively affect our measure of expertise. Therefore, it is important that we included play time as a control variable, to make sure that the expertise is the rate of accumulating achievement rather than just spending more time in the game.
The expansive nature of MMOs, coupled with the availability of myriad incentive–laden tasks results in a spread of expertise and achievement among participants, and provides a unique context for research on expertise and experts. This study is the first empirical research attempt to examine expertise and experts in the domain of MMOs by studying EverQuest II. Benefiting from the unprecedented opportunity of obtaining game log data matched with survey data, this study investigated the associations between game expertise and off–line player characteristics and game motivations and behaviors. The findings, some of which supported traditional expertise theory and some of which challenged stereotypes in gamer populations, contribute to both expertise research and game research. By proposing a performance measure of expertise in MMOs, and suggesting variable player characteristics and in–game play patterns indeed influence relative achievement, this research provides a foundation for future work.
Limitations and future research directions
This study has some limitations. First, as mentioned at the beginning, there are multiple definitions of expertise, each of which can imply a different measure. This study took the relative approach, and used the definition focusing on high performance rather than purely on high knowledge level. We consider this a more realistic measure, as expertise and performance are inextricably linked. Expertise is both most valuable and most recognized when it leads to high performance. Individuals with the mere knowledge to perform as experts, but who fail to demonstrate their abilities, invite others to doubt their expert status. In addition, this study also made sure to distinguish expertise from cumulative achievement by controlling the effect of time in the model. However, this is just one way to measure game expertise. Future research should explore more sophisticated measures of game expertise, and examine their similarities and differences.
Second, this study reports on cross–sectional data. Clearly, longitudinal research is necessary in order to determine causal relations. For instance, it is not clear based on the data reported here whether game focus leads over time to higher achievement or higher achievement leads players over time to higher game focus. Unlike other contexts, the present research program has longitudinal data from computer logs and is in the process of conducting a second wave of survey data that will enable it to address these longitudinal causal questions in future research.
Third, while all of the predictors except gender had significant effects on game expertise in this study, it is important to note that the effect sizes are mostly very small, especially compared to the intercept. Since the large sample size can have an effect on the significance results, the probability level for significance testing in this study was set at .01 rather than .05 in order to reduce possibility of false positive results. When interpreting the results, it is important to recognize that though these factors were found to have consistent, reliable effects on expertise, the effects are rather small. This may make interventions based on these factors not sufficiently effective or efficient.
Fourth, as described in the methods section, when matching the game log data and survey data, we used the primary character for each player to represent their in–game characteristics and behaviors. Dropping all the other characters may at least be inefficient, and at worst affect the results. Future research needs to think of alternative ways to deal with this issue of one player with multiple characters.
Finally, we acknowledge that our respondents self–selected to complete the survey, creating some sampling bias. Future work should include other survey recruiting methods to validate these results. We must also recognize that EverQuest II is not necessarily representative of MMOs at–large: while still popular among MMO players, and involves many of the same mechanisms found in other MMOs such as leveling, acquiring wealth and rewards, and joining with other players to complete quests or adventures, it is not nearly as popular as World of Warcraft and likely attracts a particular type of player. Future work should include players from other MMOs such as World of Warcraft, Eve, or Guild Wars.
There are four important theoretical, methodological, and practical insights to be derived from this study. First, it addresses an issue that is of perennial theoretical interest across several social scientific disciplines: what characteristics explain the development of expertise among individuals? The fact that social motivation was found to be a positive predictor of expertise underscores the critical role of communication theory in explaining the development of expertise. The study had hypothesized that those with higher social motivation would be less inclined to develop expertise. In retrospect, this hypothesis was premised on expertise development as being a solo endeavor. However, the results suggest that expertise development, which often occurs in group contexts, has a strong social and communicative component that needs to be explored further. While one should be cautious in making these claims based on this specific MMO, the results invite consideration of these issues in other virtual worlds — and indeed in helping us seek a more fundamental understanding of the development of expertise in the real world — which is increasingly both off–line and online.
Second, this study integrates several perspectives ranging from individual personality, motivational and behavioral measures to address this issue. Unlike prior research, this study is among the first to combine digital traces and self–report data to test and extend these models: the computer generated log data, not filtered by self–report biases, is appropriate for behavioral data; the survey measures not adequately captured by computer logs is appropriate for self–report perceptual measures.
Third, and related to the second point, the study showcases significant methodological accomplishments in instrumentation. This study is part of a research program that is pioneering the use of computer–based logs to instrument actions, interactions, and transactions in virtual worlds. The computational challenges for storing, managing, manipulating and inferring socially meaningful data from high-resolution computer-generated logs is a non–trivial computational challenge. The present study is unique in its effort to combine these digital traces with a large–scale survey conducted with a subset of about 6,000 participants of the virtual world. As such, this study responds to the call of researchers from different perspectives for multi–method approaches to the study of virtual worlds. While past studies have used smaller datasets to explain expertise based on self–report measures or behavioral measures (but not both), the present study provides an opportunity to combine these into a single model. This combination might well explain the somewhat surprising finding that gender was not seen as a significant predictor after accounting for the effects of other variables in the model.
Finally, the study also has important practical implications for those involved in developing these virtual worlds (What features in the virtual worlds facilitate or inhibit expertise formation?), marketing these virtual worlds (What motivates players to stay in the game and continue increasing their expertise? To what extent is level of expertise or change in the level of expertise associated with players’ decisions to stay or drop out of the game?), and training in these virtual worlds (which users are most likely to gain expertise in environments such as EQII?).
About the authors
Jing Wang’s research involves social network analysis and organizational communication. She received a Ph.D. from the Communication Department at the University of Illinois at Urbana–Champaign, and worked as a post–doctoral researcher at Northwestern University.
David Huffaker is a researcher at Google, Inc. His research focuses on communication and networks in social media.
Jeff Treem is a Ph.D. student in the Media, Technology, and Society program in Northwestern University’s School of Communication. His research interests include the relationship between technology use and social perceptions of expertise.
Lindsay Fullerton is a Ph.D. student in the Media, Technology and Society program in the School of Communication at Northwestern University.
Muhammad A. Ahmad is a Ph.D. student in the Department of Computer Science and Engineering at the University of Minnesota (UMN).
Dmitri Williams is an associate professor at the USC Annenberg School for Communication & Journalism, where he is a part of the Annenberg Program on Online Communities (APOC).
Marshall Scott Poole is David and Margaret Romano Professorial Scholar, Senior Research Scientist at the National Center for Supercomputing Applications, and Director of I–CHASS: The Institute for Computing in the Humanities, Arts, and Social Sciences at the University of Illinois.
Noshir Contractor is the Jane S. & William J. White Professor of Behavioral Sciences in the McCormick School of Engineering & Applied Science, the School of Communication and the Kellogg School of Management at Northwestern University. He is the Director of the Science of Networks in Communities (SONIC) Research Group.
The research reported herein was supported by the National Science Foundation via award number IIS–0729505, the Army Research Institute via award number W91WAW–08–C–0106, and the Air Force Research Laboratory by contract number FA8650–10–C–7010. The data used for this research was provided by Sony Online Entertainment. We also thank members of the Virtual Worlds Observatory team for their support and feedback.
Author disclosure statement
No competing financial interests exist.
1. D. Williams N. Yee N, and S.E. Caplan, 2008. “Who plays, how much, and why? Debunking the stereotypical gamer profile,” Journal of Computer–Mediated Communication, volume 13, number 4, pp. 993–1,018.
2. P.G. Schrader and M. McCreery, 2008. “The acquisition of skill and expertise in massively multiplayer online games,” Educational Technology Research and Development, volume 56, numbers 5–6, pp. 557–574.
3. E. Castronova, 1995. Synthetic worlds: The business and culture of online games. Chicago: University of Chicago Press.
4. M. Griffiths, M.N. Davies, and D. Chappell, 2003. “Breaking the stereotype: The case of online gaming,” CyberPsychology & Behavior, volume 6, number 1, pp. 81–91.http://dx.doi.org/10.1089/109493103321167992
5. N. Yee, 2006. “The demographics, motivations, and derived experiences of users of massively multi–user online graphical environments,” Presence, volume 15, number 3, pp. 309–329.http://dx.doi.org/10.1162/pres.15.3.309
6. B. Reeves, T.W. Malone, T. O’Driscoll, 2008. “Leadership’s online labs,” Harvard Business Review, volume 86, number 5, pp. 58–66.
7. K.A. Ericsson, R.T. Krampe, and C. Tesch–Römer, 1993. “The role of deliberate practice in the acquisition of expert performance,” Psychological Review, volume 100, number 3, pp. 363–406.http://dx.doi.org/10.1037/0033-295X.100.3.363
8. J. Bedard and M.T.H. Chi, 1992. “Expertise,” Current Directions in Psychological Science, volume 1, number 4, pp. 135–139.http://dx.doi.org/10.1111/1467-8721.ep10769799
9. H. Collins and R. Evans, 2007. Rethinking expertise. Chicago: University of Chicago Press.
10. K.A. Ericsson, 2005. “Recent advances in expertise research: A commentary on the contributions to the special issue,” Applied Cognitive Psychology, volume 19, number 2, pp. 233–241.http://dx.doi.org/10.1002/acp.1111
11. H.A. Simon and W.G. Chase, 1973. “Skill in chess: Experiments with chess–playing tasks and computer simulation of skilled performance throw light on some human perceptual and memory processes,” American Scientist, volume 61, number 4, pp. 394–403.
12. H.A. Mieg, 2006. “Social and sociological factors in the development of expertise,” In: K.A. Ericsson, N. Charness, P.J. Feltovich, and R.R. Hoffman (editors). The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press, pp. 743–760.
13. M.T.H. Chi, 2006. “Two approaches to the study of experts’ characteristics,” In: K.A. Ericsson, N. Charness, P.J. Feltovich, and R.R. Hoffman (editors). The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press, pp. 21–30.
14. R. Bartle, 1996. “Hearts, clubs, diamonds, spades: Players who suit MUDs,” Journal of Virtual Environments, volume 1, number 1, at http://www.mud.co.uk/richard/hcds.htm, accessed 12 April 2010.
15. N. Yee, 2007. “Motivations of play in online games,” CyberPsychology & Behavior, volume 9, number 6, pp. 772–775.http://dx.doi.org/10.1089/cpb.2006.9.772
16. M. Griffiths, M.N. Davies, and D. Chappell, 2004. “Demographic factors and playing variables in online computer gaming,” CyberPsychology & Behavior, volume 7, number 4, pp. 479–487.http://dx.doi.org/10.1089/cpb.2004.7.479
17. K. Lucas and J.L. Sherry, 2004. “Sex differences in video game play: A communication–based explanation,” Communication Research, volume 31, number 6, pp. 499–523.http://dx.doi.org/10.1177/0093650204267930
18. J. Cassell and H. Jenkins, 1998. “Chess for girls? Feminism and computer games,” In: J. Cassell and H. Jenkins (editors). From Barbie to Mortal Kombat: Gender and computer games. Cambridge, Mass.: MIT Press, pp. 2–45.
19. J.D. Ivory, 2006. “Still a man’s game: Gender representation in online reviews of video games,” Mass Communication & Society, volume 9, number 1, pp. 103–114.http://dx.doi.org/10.1207/s15327825mcs0901_6
20. J.L. Sherry, K. Lucas, B.S. Greenberg, and K. Lachlan, 2006. “Video game uses and gratifications as predictors of use and game preference,” In: P. Vorderer and J. Bryant (editors). Playing video games: Motives, responses, and consequences. Mahwah, N.J.: Lawrence Erlbaum Associates, pp. 213–224.
21. D. Williams, M. Consalvo, S. Caplan, and N. Yee, 2009. “Looking for gender (LFG): Gender roles and behaviors among online gamers,” Journal of Communication, volume 59, number 4, pp. 700–725.http://dx.doi.org/10.1111/j.1460-2466.2009.01453.x
22. P.L. Ackerman, 1996. “A theory of adult intellectual development: Process, personality, interests, and knowledge,” Intelligence, volume 22, number 2, pp. 227–257.http://dx.doi.org/10.1016/S0160-2896(96)90016-1
23. R.J. Sternberg, 1998. “Abilities are forms of developing expertise,” Educational Researcher, volume 27, number 3, pp. 11–20.http://dx.doi.org/10.3102/0013189X027003011
24. P. Vorderer, T. Hartmann, and C. Klimmt, 2003. “Explaining the enjoyment of playing video games: The role of competition,” ICEC ’03: Proceedings of the Second International Conference on Entertainment Computing (Pittsburgh, Penn.), pp. 1–8.
25. E. Hunt, 2006. “Expertise, talent, and social encouragement,” In: K.A. Ericsson, N. Charness, P.J. Feltovich, and R.R. Hoffman (editors). The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press, pp. 31–38.
26. J.P. Gee, 2003. “What video games have to teach us about learning and literacy,” Computers in Entertainment, volume 1, number 1, pp. 1–4.http://dx.doi.org/10.1145/950566.950595
27. C.A. Steinkuehler, 2006. “Massively multiplayer online video gaming as participation in a discourse,” Mind, Culture, and Activity, volume 13, number 1, pp. 38–52.http://dx.doi.org/10.1207/s15327884mca1301_4
28. H. Cole and M. Griffiths, 2007. “Social interactions in massively multiplayer online role–playing games,” CyberPsychology & Behavior, volume 10, number 4, pp. 575–583.http://dx.doi.org/10.1089/cpb.2007.9988
29. A.F. Seay, W.J. Jerome, K.S. Lee, and R. Kraut, 2003. “Project Massive 1.0: Organizational commitment, sociability and extraversion in massively multiplayer online games,” Proceedings of Level Up 2003 Conference of the Digital Games Research Association (Utrecht, Netherlands), at http://www.projectmassive.com/publications.html, accessed 15 July 2011.
30. C.A. Anderson and B.J. Bushman, 2001. “Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A Meta–analytic review of the scientific literature,” Psychological Science, volume 12, number 5, pp. 353–359.http://dx.doi.org/10.1111/1467-9280.00366
31. D. Williams and M. Skoric, 2005. “Internet fantasy violence: A test of aggression in an online game,” Communication Monographs, volume 72, number 2, pp. 217–233.http://dx.doi.org/10.1080/03637750500111781
32. S. Kim, 2001. “Designing Web games that make business sense,” paper presented at the 2001 Game Developers Conference (San Jose, Calif.), at http://www.scottkim.com/thinkinggames/GDC01/webgamesbusinesssense.html, accessed 15 July 2011.
33. A.H. Buss and M. Perry, 1992. “The aggression questionnaire,” Journal of Personality and Social Psychology, volume 63, number 3, pp. 452–459.http://dx.doi.org/10.1037/0022-35184.108.40.2062
34. A.W. Bendig, 1962. “The Pittsburgh scales of social extraversion–introversion and emotionality,” Journal of Psychology, volume 53, pp. 199–209.http://dx.doi.org/10.1080/00223980.1962.9916565
Survey questions. For motivation variables: When you play EverQuest II, how important are the following aspects of game-play to you? Not important at all ::1 Slightly important ::2 Somewhat important ::3 Very important ::4 Extremely important ::5 Don’t know ::99 (recoded as System Missing) import1 Leveling, acquiring great items and gear, and becoming powerful. import2 Figuring out the game mechanics, planning my character's development, and optimizing my character. import3 Competing with other players in terms of combat, crafting ability, or the economy. import4 Chatting with and getting to know other players. import5 Developing deep and meaningful relationships with other players. import6 Having a character that can solo well and work independently. import7 Being part of a team. import8 Exploring the world and knowing things (stories, locations of NPCs, etc.) that most other players don’t know about. import9 Role–playing and having interesting background stories for your character. import10 Customizing your characters to make them look distinctive, stylish, and unique. import11 Escaping from the real world and leaving behind some real–life problems and worries. For the aggression (aqp) and extroversion (social) scales Below are several statements that may represent beliefs you have about yourself. To what extent do you agree or disagree with them? 7 Strongly Agree ::7 6 ::6 5 ::5 4 ::4 3 ::3 2 ::2 1 Strongly Disagree ::1 Don’t know :: 99 (recoded as System Missing) social1 I often feel as though I don’t have a care in the world. aqpa7 I can think of no good reason for ever hitting a person. social3 I usually take the initiative in making new friends. aqpa1 Once in a while I can’t control the urge to strike another person. social5 Other people regard me as a lively individual. rsoc6 I do not enjoy the excitement of a crowd. social7 I would be very unhappy if I were prevented from making numerous social contacts. rsoc8 I’m usually quiet in a group of people. rsoc9 I would not rate myself as a talkative individual. rsoc10 I usually keep in the background on social occasions. aqpa3 If somebody hits me, I hit back perb11 I often get involved in something so deeply that nothing else seems to matter and I lose track of time. aqpa5 If I have to resort to violence to protect my rights, I will. aqpa8 I have threatened people I know. aqva2 I often find myself disagreeing with people. aqva5 My friends say that I’m somewhat argumentative. rsoc4 I seldom have a “ready answer” for remarks directed at me. aqpa6 There are people who pushed me so far that we came to blows. aqpa9 I have become so mad that I have broken things. aqpa2 Given enough provocation, I may hit another person. aqpa4 I get into fights a little more than the average person. aqva4 I can’t help getting into arguments when people disagree with me. aqva3 When people annoy me, I may tell them what I think of them. social2 I like to mix socially with people. aqva1 I tell my friends openly when I disagree with them. Reverse coded items have been flipped and turned into the right direction prior to scale construction. For example, “rsoc4” was flipped and turned into “social6” The original values in the “r” versions are still in the data file.
Received 7 July 2011; revised 13 July 2011; accepted 15 July 2011.
Copyright © 2011, First Monday.
Copyright © 2011, Jing Wang, David A Huffaker, Jeffrey W. Treem, Lindsay Fullerton, Muhammad A. Ahmad, Dmitri Williams, Marshall Scott Poole, and Noshir Contractor.
Focused on the prize: Characteristics of experts in massive multiplayer online games
by Jing Wang, David A Huffaker, Jeffrey W. Treem, Lindsay Fullerton, Muhammad A. Ahmad, Dmitri Williams, Marshall Scott Poole, and Noshir Contractor.
First Monday, Volume 16, Number 8 - 1 August 2011