Estimating online audiences: Understanding the limitations of competitive intelligence services

David Kamerer

Abstract


How much traffic does a Web site receive? While many may want to know the relative influence of someone else's site, generally only the site owner has access to accurate data. JavaScript-based Web analytics, such as Google Analytics or Omniture, are widely considered the gold standard for tracking audiences. These services are census- rather than sample-based. They also have the virtues of being entirely passive (requiring no input from the visitor) as well as providing true measures of behavior.

If you're not the site owner, you'll have to use competitive intelligence services to estimate the traffic. These services a wide range of methodological approaches, generally recruiting panel members using an opt-in mechanism, sometimes mixing in anonymized data bought from Internet service providers. Three of the leading free services of this type are Alexa, Compete, and Quantcast.

This study examines the methodology of these four methods of measuring online audiences. It compares the results from all three services using a panel of 18 Web sites that have agreed to share analytics data with the author.

Results show high variability in the quality of the estimated audience data from competitive intelligence services. Using analytics as the reference, Compete underestimated traffic by more than 300 percent. Quantcast underestimated traffic by more than 200 percent. Confounding this pattern were occasional overestimates of traffic by these services. Alexa data, which is presented differently than Compete and Quantcast data, showed one significant correlation (page views) with analytics data, while three other variables showed no relationship to data from analytics.

Overall, these services performed poorly. But are the data worthless? The report concludes with recommendations for understanding the limitations of competitive intelligence services and appropriately reporting audience activity based upon differing methodologies.


Keywords


analytics; methodology; audience behavior; audience research

Full Text:

HTML


DOI: http://dx.doi.org/10.5210/fm.v18i5.3986



A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2017. ISSN 1396-0466.