Undermining 'data': A critical examination of a core term in scientific inquiry
First Monday

Undermining 'data': A critical examination of a core term in scientific inquiry by Annette N. Markham



Abstract
The term ‘data’ functions as a powerful frame for discourse about how knowledge is derived and privileges certain ways of knowing over others. Through its ambiguity, the term can foster a self–perpetuating sensibility that ‘data’ is incontrovertible, something to question the meaning or the veracity of, but not the existence of. This article critically examines the concept of ‘data’ within larger questions of research method and frameworks for scientific inquiry. The current dominance of the term ‘data’ and ‘big data’ in discussions of scientific inquiry as well as everyday advertising focuses our attention on only certain aspects of the research process. The author suggests deliberately decentering the term, to explore nuanced frames for describing the materials, processes, and goals of inquiry.

Contents

1. Introduction: Data is a deceptively easy term to toss around
2. ‘Data’ functions as a frame for mapping
3. Frames for ‘everyday life’ influence the way researchers think about inquiry
4. Finding nuanced frameworks can help build adaptive approaches for studying complexity
5. Conclusion: From matters of fact to matters of concern

 


 

1. Introduction: Data is a deceptively easy term to toss around

The term ‘data’ was problematized early in my academic career. Studying interpretive sociology and engaging in post–modern discussions of ethnography in the early 1990s, I was taught to use any word except data. Gradually the word seeped back into my vocabulary and I now use it with abandon. For me, it functions as a metonym to mean “the stuff I’m focusing on as I explore this phenomenon.”

Data is, as research terminology goes, a deceptively easy word to toss around. It’s easily accessible for most of us, fills in as a better descriptor than the term ‘stuff,’ and adds instant credibility to that which it describes. The term ‘data’ does far more than describe units of information used in the course of one’s study. It functions as a powerful frame for discourse about knowledge — both where it comes from and how it is derived; privileges certain ways of knowing over others; and through its ambiguity, can foster a self–perpetuating sensibility that it is incontrovertible, something to question the meaning of, or the veracity of, but not the existence of.

As a research term, ‘data’ has been a problem for qualitative researchers for some decades now, not least because the term is — in most common usage — associated with some thing that one gathers, hence is a priori and collectable. Data are potentially informational, yes, but as operationalized in most of the social or natural sciences, function fundamentally as discrete objects that can be located in time and space. The problem with this conceptualization is that it remains categorically different from — and in a sense opposed to — the very idea of process. From a qualitative perspective, ‘data’ poorly capture the sensation of a conversation or a moment in context.

The rise of the term ‘big data’ has brought many critical responses, including the idea that data is always big, ethnography is always big data, small data are important, and so forth. As many have emphasized, ‘big data’ is less about the characteristics of data themselves than the shift to computational tools and methods for analyzing large sets of information. Recent arguments that big data mean nothing without context (boyd and Crawford, 2012) or that the very idea of ‘raw data’ is an oxymoron (Bowker, 2005; Gitelman, 2013) provide timely and subtle responses.

I agree with these critical responses, but shift my own angle of gaze to forms of knowledge production more generally, asking: What is involved in everyday inquiry, anyway? This is an intriguing question, because to grapple with it requires reflection both above and below questions of method, at least in academic circles. Above method, we can consider the epistemological, if not ontological conditions within which we find ourselves doing social research in the twenty–first century, which are heavily influenced by our disciplinary histories and infrastructures for inquiry. These are also not completely separated from political and economic considerations, such as competition for shrinking pools of funding, regulation of research by ethics review boards, and closer scrutiny from taxpaying publics who struggle in tough economic times to justify research that does not tackle social problems through evidence–based scientific practice. Below method means taking a closer critical look at what we actually do when we’re engaging in inquiry, scientific or otherwise. By deliberately dismissing the disciplined terminology that goes along with ‘scientific inquiry’ or ‘academic inquiry’ and simply focusing on what good inquiry involves at the level of everyday practice, we can witness a complex, generative process of sense–making that in its best sense, eludes reduction or simplification.

 

++++++++++

2. ‘Data’ functions as a frame for mapping

As an interpretive communication scholar trained in a tradition of pragmatism, I use a range of approaches that could be grouped in the Chicago school of sociology. As someone who studied theorists like Gregory Bateson, Kenneth Burke, and Erving Goffman, I can’t help but reflect on this issue by thinking about frames. What are the frameworks or parameters surrounding our research practices in the social sciences and humanities? [1] Consider the frame the term ‘data’ constructs around our practices as curious people wanting to make sense of the world around us, whether we’re designing new technology interfaces, working to improve the bottom line, investigating a crime scene, teaching the new neighbors how to garden, or, closer to my own home, conducting social research in the humanities or social sciences.

The interesting thing about frames, as social psychologist Goffman (1974) noted, is that they draw our attention to certain things and obscure other things. In this traditional type of frame (Figure 1), for example, we’re supposed to attend to what’s inside the frame, blocking out what’s outside. If I add two simple words to your view, such as ‘Mona Lisa,’ you will likely begin to imagine this famous painting, inside this frame. If I ask you to consider what is outside this frame, you likely visualize a wall and if you’ve been to the Louvre where this painting hangs, you might imagine the iconic glass pyramids situated just outside the entrance to the museum. If it’s been your screensaver on a desktop, you might imagine your own walls, of course, but the point of the frame in demarcating a clear figure and ground is clear.

 

picture frame
 
Figure 1.

 

Now consider Figure 2 below. If I tell you this is a frame, it certainly changes how we might think about what might constitute a frame, as well as how frames direct our attention. Compared to Figure 1 as a tool for drawing attention to particularities of a situation or phenomenon like ‘mona lisa,’ this sphere shifts what we pay attention to.

 

globe
 
Figure 2.

 

One might begin to see dimension, an inside, an outside, and perhaps movement. There’s less definite inside/outside and more implied motion in space. Is this a sketch of the planet Earth, where a painting might be pinpointed in one geographical area (Paris, France)? If so, this idea might focus our attention on location–related details, rather than characteristics of the painting itself. Maybe we could look at how the painting moved over time around this globe. Maybe we could consider how various countries around the world have integrated the painting of the ‘Mona Lisa’ into their everyday nomenclature. The possibilities are interesting, but let’s shift the frame again to Figure 3.

 

model
 
Figure 3.

 

With this frame, we might note that everything (almost) is interconnected. We’re no longer seeing (and perhaps not as concerned with) what’s inside or outside, as with Figure 1, but now focus on a network that includes elements (nodes) and connections among them. If this is also ‘mona lisa’ we might consider relationships, connections between this artwork and other representations of women at the time, links between elements of Leonardo’s technique in this painting and his other works.

There may be information in these frames themselves to give a clue as to how they function, but in general, frames are difficult to identify because most of the time, as Goffman (1974) notes, they function invisibly in specific social contexts to shape our behavior. They work best when they are absorbed and forgotten. Taken for granted social and cultural structures take on obdurate qualities over time, as our responses to situations shift from choices to routines, from habits to naturalized ways of doing things. This occurs not only in physical behaviors, like naturally knowing to turn a handle to open a door, but in the way we sort, categorize, and make sense all aspects of our social worlds. Rhetorical theorist Kenneth Burke (1966) focuses on how this occurs through language practices that through various processes of classification and categorization create ‘terministic screens’ through which our understanding of the world is filtered (which I always imagine to be giant sieves around our heads). Conceptual metaphor theorists (e.g., Lakoff and Johnson, 1980; Reddy, 1979) would focus on the connection between our habitual language use and how we make sense of the world. ‘Information superhighway,’ for example, was a useful metaphor at one stage of the development of the Internet, but is largely unused now. Lakoff and Johnson (1980) would say this conceptual metaphor is absorbed and no longer noticed. But the root of this metaphorical comparison remains: Information is something that can be transported along a conduit called the Internet. Using Goffman’s notion of frames, this visualization calls up certain mental images such as pipelines, roads, tunnels, or other types of passageways and thoroughfares. These images tend to stick. Over time, they build a primary framework for making sense of the Internet and information. We don’t much see the Internet as a wall or cliff, for example, which might explain why the concept of the ‘digital divide’ never developed a strong foundation for sense–making among the general public. A primary framework of “Internet as conduit” literally and figuratively visualizes information as bits of stuff that travel very quickly between two points. In this way, we can exchange money, retrieve information, and deliver education. These metaphors are powerful; they can’t help but influence the way we act within this set of frames. Alternate characteristics of information, which might lie outside its conceptualization as an object that can be transported intact from one place to another, are minimized.

In short, whether we use ‘frame,’ ‘metaphor,’ ‘terministic screen,’ or some other term, when we use words that stand in for other words, this function of language not only provides a shortcut, but reinforces particular mental images, framing experience in a particular, political, and powerful way. They can literally make us see our world differently.

To mix in other metaphors, frames function like maps, orienting us to our own and others’ positions and guiding our movements. We don’t notice these frames unless they’re disrupted by some anomalous event that Goffman would call a frame break. Or we can reveal underlying frames by deliberately breaking rules or norms, as sociologist Harold Garfinkel (1967) encouraged his students to do in elevators. Or we can redraw maps in different ways, so as to argue that — and perhaps how — a frame was functioning in the first place, as we see in McArthur’s universal corrective map of the world or the Peters projection world map.

As a term within a particular frame and a frame itself, ‘data’ has high ambiguity, which, when combined with the illusion of shared understanding, can function to make us all think we’re looking at the same map when we’re not. For some who read my work, the term might not be understood as a shortcut term standing in for a complex web of contextual meaning in which I have a particular situated position. Rather, ‘data’ could be taken as the pre–existing aspects of the phenomenon, which have been collected, funneled through some hypotheses, sorted, analyzed, categorized, and reproduced faithfully as ‘findings.’

This is the sort of disconnect that disturbs my sleep. It’s also the sort of disconnect that seeps into conversations where it is assumed that everyone has a mutual understanding of what ‘data’ means, and this foundation is then taken for granted. I was asked recently to consider: How can we make qualitative research more important in the arena of big data? If big data is the purview of quantitative and computational analysts and qualitative researchers don’t want to be left behind, how can they better inform big data research and researchers? I had trouble addressing these questions because they begin from faulty premises. Most definitions of ‘big data’ these days work from very different baseline assumptions than I would as an ethnographer. That doesn’t mean ethnographic data is not ‘big.’ As Boellstorff (2012) puts it, ethnography has always been about big data, if complexity is a measure of size. But a computational analyst and an interpretive analyst likely think very differently about the ‘stuff’ of our inquiry. The two positions may not be incommensurate in all ways, but at the basic level of qualitative versus quantitative methods, they certainly are.

 

++++++++++

3. Frames for ‘everyday life’ influence the way researchers think about inquiry

What are some of the everyday frames influencing the way we think about people, the social world, and the nature of reality? This is certainly not a new question, but it’s worth reminding ourselves that as scientists, we are not immune to the influence of common frameworks for thinking about everyday life. Stepping back from the specific concept of data, consider some of the more general metaphors of everyday life in the twenty–first century. At least this decade, our most visible frames of meaning draw attention to the digital and informational qualities of whatever we’re looking at, whether it’s an advertisement, movie, journalistic or scientific explanation of a current event, or a self–help book. I turn to a very minor example to illustrate the pervasiveness of a particular version of this frame whereby everything, and I mean every aspect of human existence, is transformed and equalized as a unit or bit of information.

In this 2011 advertisement for the Samsung Galaxy II smartphone (Figure 4), tiny objects swarm around the central person in a video (at http://www.youtube.com/watch?v=vWTPNBvHFnY):

 

tiny objects
 
Figure 4 Images from 2011 ad for Samsung Galaxy II [2].

 

During this commercial, the narrator tells us:

Your life is a galaxy. Made up of a million things. Things you need, things you want. Your work. And play. Sounds, dreams, moments, everything that makes life so full of life. This is your galaxy. And it lives in the revolutionary new Samsung Galaxy SII. The fastest, the brightest, the sharpest, the greatest smartphone ever. Own the new Samsung Galaxy S II. Be the master of your universe.

This advertisement is interesting on many levels. It presents a particular way of seeing the social and natural world. It flattens all experiences, reducing and then equalizing people, emotions, and dreams to the same level as goods and services. It then chops everything into tiny but not infinitesimal units, all of which can be transmitted and more importantly controlled — well, as long as we purchase the smartest phone, which is one that can transmit these units quickly and accurately. This framework is not new, but a different iteration of a gradual shift in thinking from atoms to bits, as Negroponte (1995) noted.

We might not consider this anything but a part of a particular company advertising a particular product, using a particular advertising campaign strategy. But it functions well beyond this purpose to encapsulate a common way of thinking about the nature of being in the twenty–first century. It exists within a much larger context whereby experience is simultaneously displayed as diverse, immense, and complex, flowing in a global network of connections, but also collapsed into collectable data points. Once this conceptual move is made — to transform experience into digital information, it is only a small step further to visualize it as a series (or assemblage) of objects that necessarily have obdurate qualities that can be collected, sorted, categorized, and analyzed. Within a framework where experience is digitized, it’s not surprising that we think of everything as data.

Again, it’s important to note that this is not a new way of thinking, but another plotline within a long story about how we characterize, study, and therefore ‘know’ the nature of the world around us, including ourselves. Any number of philosophers could (and do) articulate this story better than I could ever hope to. Bruno Latour (1999), for instance, uses a study of dirt in the Amazon to illustrate the ways in which particular conceptions of data pervade scientific research, effectively transforming reality into an abstraction called ‘data’ that reduces complexity to such a radical degree that it no longer resembles anything like that which it is supposed to represent. In the same spirit as philosophers of science and technology, I use this Samsung ad to illustrate a pervasive and troubling conception of “humans (and their data) as data,” which troubles me in the same way it bothers Grinter (2013). This ‘datafication’ transgresses advertising, everyday life, and scientific practice. The question is not only what might we be missing, in our attempt to capture and encapsulate human experience in data trails (as Baym discusses in more depth in this issue), but what are we losing sight of in the larger picture of the consequences of thinking in this way about human and social experience?

 

++++++++++

4. Finding nuanced frameworks can help build adaptive approaches for studying complexity

If we can recognize how frames like these function to construct the parameters of our everyday social experiences, we gain leverage in critically examining how “scientific inquiry” is being delimited within this framework. We can also continue to push alternative conceptualizations.

This discussion, as it relates to big data, is taken up by many others, including the provocative critical work by boyd and Crawford (2012), authors in the edited collection by Gitelman (2013), and other authors in this special issue (e.g., Boellstorff, Halavais). Here, I shift to the level of research methods to consider what alternate frames might be brought to the foreground in developing creative and innovative approaches to inquiry in the so–called digital age. I offer two that have been helpful in my own grounding as an ethnographer who uses a mix of interpretive, mostly qualitative methods to analyze social media phenomena and lived experience in digitally mediated culture.

Of course, any alternate framework cannot be all–inclusive or provide the answer to any perceived or actual limitation, when it comes to research methods. Frames should be considered useful provocations, meant to invite conversation about the differences among our goals, premises, and methods for inquiry. Among these two I offer, inquiry as generative and inquiry as collaborative remix, data plays a role, but this role is more like the chorus than the lead. The premises are well represented by Geertz’s descriptions of ethnography as an interpretive act of ‘thick description’:

[T]his fact — that what we call our data are really our own constructions of other people’s constructions of what they and their compatriots are up to — is obscured because most of what we need to comprehend a particular event, ritual, custom, idea, or whatever is insinuated as background information before the thing itself is directly examined ... . There is nothing particularly wrong with this, and it is in any case inevitable. But it does lead to a view of anthropological research as rather more of an observational and rather less of an interpretive activity than it really is. Right down at the factual base, the hard rock, insofar as there is any, of the whole enterprise, we are already explicating: and worse, explicating explications. Winks upon winks upon winks. [3]

4.1. Consider the generative aspects of inquiry

All inquiry is intensely generative. From a very practical look at the process of research, we can see data being generated every step of the way. It’s not just that one question leads to others, or that an initial curiosity will evolve into a full–blown investigation. Anything that functions as a source of information is constantly being generated through the choices we make along the way. This has ethical and practical and knowledge–making consequences because we are constantly, through our everyday actions, transforming the phenomenon under study.

Anthropologist Sarah Pink (2012), drawing on phenomenological premises, addresses this issue by extending the stages and senses of inquiry to include not only what is present in the most obvious ‘collected’ or ‘collectable’ sense but to also look at what is done before, during, after, and between. Jackson (1990) explores the way that field notes function on multiple levels and construct layer upon layer of observations, thoughts, analytical points, theoretical meanderings, reconstructions of events, records of memories, sketches, and a variety of what might look like chicken scratching, all of which factors into how the ethnographer is producing meaning, or as Clifford and Marcus (1986) so famously put it, writing culture.

The question emerging after the post–modern era is perhaps not to ask whether research is a generative process, but how to incorporate this idea into one’s methods without losing fidelity and veracity. It may be a matter of shifting the question slightly. Law and Mol (2002) take for granted the generative qualities of inquiry and instead address the issue of whether and how one ought to simplify or embrace complexity. Among other things, they emphasize that simplicity is not necessarily opposite of complexity. They suggest that some form of simplification will always occur, so “it becomes instead a matter of determining which simplification or simplifications we will attend to and create, and as we do this, of attending to what they foreground and draw our attention to, as well as what they relegate to the background” [4].

It may also be an issue of shifting the goal of inquiry, from one that seeks stability or order to one that seeks to compel, relate, or explore, understanding the inherent open-endedness of this act in contextual space and time. The key would be to add transparency, acknowledging that one is engaging in sense–making rather than discovering or finding or attempting to classify in a reductionist sense.

Looking more directly at processes and procedures that underlie most social science research, we situate ourselves in relation to a phenomenon. The generative qualities of inquiry change depending on where we find ourselves in the research process. Every field note produces not a description of the field but a new form of data to be interpreted. A coffee stain on the research journal can spark a visceral memory previously forgotten. The scent on the wind, a dream, a drive past another neighbor’s garden; all these are part of the ways we make sense of everyday life, and certainly provoke particular perspectives on whatever phenomenon we are studying. As we write, we generate iteration after iteration of interpretations, renderings as valuable as an architect’s or artist’s, but often discarded as doodling, brainstorming, or merely ‘thinking through’ an idea.

This generative process often happens without the cognizance of the researcher. This has consequences. For example, when using social networking visualization software like Gephi, a simple click of a button will apply an algorithm to ‘clean up’ one’s visualization, giving a network map a more tidy appearance. One can drag nodes around to make certain nodes bigger or smaller, highlighting different relations. This is frequently dismissed as a non–analytical activity when in actuality it constitutes an important form of play that can, through the generation of new data maps, sponsor different insights and change the fundamental way in which the data is seen from that point forward (Markham and Lindgren, in press).

It’s not a surprise to me that we don’t notice, or even tend to dismiss this notion. Academic reports are presented as smooth overviews or explanations; findings are often presented as conclusions, not conversation starters. The entire industry of academic publishing is predicated on the foundation of knowledge building, which looks more like a finished product than an ongoing dialogue among colleagues. The typical form of an academic article (which I try to counter somewhat here) is intended to wrap things up, simplify rather than complexify, and be experienced, as Weinberger notes below, as a closed system. Understanding that we are always making more ‘data’ every time we think about or engage in the project or the context is one thing. Attempting to capture and include these as data is an impossibility that Alice in Wonderland might face. Still, as I have noted elsewhere, (e.g., Markham, 2005; Markham, 2012), the complexity of twenty–first century culture requires finding perspectives that challenge taken for granted methods for studying the social in a digital epoch. Contributing to an infrastructure of knowledge that does not reduce or simplify experience requires us to acknowledge and scrutinize, as part of our methods, the ways in which data is being generated (we are generating data) in ways we may not notice. Changing the frame from one that is overly–focused on ‘data’ can help us explore the ways our research exists as a continual, dialogic, messy, entangled, and inventive process when it occurs outside the walls of the academy, the covers of books, and the written word.

4.2. Consider inquiry as collaborative remix

David Weinberger suggests that “the networking of knowledge may be teaching us that the world itself is more like a shapeless, intertwingled, unmasterable web than like a well–reasoned argument” [5]. When I put my teacher hat on, this presents many complications. How can one manage to instruct research methods as a reasoned, logical process, involving somewhat linear stages, with a definite beginning and ending point? The easy answer is that one can use any typical methods textbook where these ideas are laid out already. As I’ve noted previously, “despite the influence of feminist, postmodern, social constructivist, or post positivist thinking, top–down models for social research continue to converge with hard science interpretations, creating an environment that continues to privilege conventional, standardized procedures for the proper conduct of research and form of writing research findings” [6]. We might teach critical reflexivity to combat the tendency toward oversimplification and reductionism, but this doesn’t really address the crux of the illusion.

We’re not just asking the wrong question. We’re using frames that both privilege and reinforce a very narrow notion of inquiry. I find remix to be a productive continuation of a strong lineage of efforts to reconceptualize research processes, products, and goals [7]. A remix conceptualization of inquiry emphasizes that any articulation of knowledge is a process of finding, borrowing, and sampling from any number of relevant sources, creatively reimagining how these elements might be put together, and then creating an assemblage that one hopes has significance, salience, and meaning for those people who experience it.

To borrow from my own description of Remix,

The history of Remix is most often linked to the work [of] generations of hip hop artists engaged in dub, scratch, rap, and DJ, [who] began deconstructing and reconstructing musical tracks in the late 60s. We’re now very familiar with the way songs are remixed in ways that extend or reinterpret them for different audiences. But remix goes well beyond music.

Remix has become a term that is used to describe the widespread practice of mashup videos, most evident on YouTube, or the phenomenon of Internet memes, which are typically composed of small units of cultural information (a phrase, an image, a short audio or video clip) that get mixed in different ways, generally for comedic effect. A meme is characterized by its evolution — in effect, it doesn’t exist unless it morphs through reproduction and dissemination.

We could say Remix is everywhere, or “everything is a remix” (Ferguson, n.d.), as both a practice and outcome in all forms of cultural production ... Lessig (2008) and Ferguson (n.d.) ... argue that it’s the content of an idea, not the originator, that matters, and that borrowing, sampling, and creatively remixing ideas is an inherent aspect of any culture ... . Remix is not something we do in addition to our everyday lives, it is the way we make sense of our world. [8]

Whether for parody, criticism, argument, or art, remix succeeds when it moves. By this I mean how it moves as well as how it moves us. Remix implies change. It only exists as a response to something else, acknowledging that its meaning is derived only in relation to its various referents, whether these are embedded in the remix itself or the context in which it is experienced. Remix is intended to move literally through networks, getting passed around and morphing into many other pieces and moments. Remix is also meant to move us — get us to pay attention to it, push us emotionally, compel us to think or act differently.

Using remix as a lens for thinking about research is intended to destabilize both the process and products of inquiry, but not toward the end of chaos or “anything goes.” The idea of remix simply refocuses energy toward meaning versus method; engagement versus objectivity; interpretation versus findings; argument versus explanation. In all of this, data is certainly available, present, and important, but it takes a secondary role to sense–making. Remix also resonates well with messy, networked, global, contexts. It tries to find a sensible pathway to meaning in a world whereby, as Weinberger aptly notes,

The final product of science is now neither final nor a product. It is the network itself — the seamless connection of scientists, data, methodologies, hypotheses, theories, facts, speculations, instruments, readings, ambitions, controversies, schools of thought, textbooks, faculties, collaborations, and disagreements that used to struggle to print a relative handful of articles in a relative handful of journals. [9]

 

++++++++++

5. Conclusion: From matters of fact to matters of concern [10]

It is difficult to speak outside the symbolic frames of positivism or empiricism. Anything described as anti–positivist still centralizes positivism. Without deliberate efforts to shift the grounds of discussion, positivist premises linger. Rosenberg’s (2013) discussion of data provides a powerful explication of how the complexity of the situation or what Geertz describes above as ‘background information,’ gets lost when the term data is used as a metonym. As Rosenberg describes, the definition of the term data shifted significantly over the past 300 years. Originating from the Latin ‘dare,’ or ‘to give,’ datum was something that was granted as given in an argument [11]. The term gradually shifted from a description of that which precedes argument to that which is pre–analytical and pre–semantic. Put differently, data is beyond argument. It always exists, no matter how it might be interpreted. Data has an incontrovertible ‘itness.’ As Rosenberg writes, its rhetorical function is very strong: “When a fact is proven false, it ceases to be a fact. False data is data nonetheless” [12]. Over time, the term has come to represent the entirety of what the researcher seeks and more importantly, needs, in order to discover, which is seen as the ultimate goal of inquiry. Deliberately shifting the frame by focusing on other aspects or goals of inquiry can be a fruitful means of exploring alternatives. By clearing away some of the data fog enveloping and somewhat smothering our sensibilities about the potentials of inquiry, we can bring to the foreground other salient and critical elements of the research situation.

The two frames I’ve mentioned here, among many others, can help us grapple with the impossible challenge of endless — that is, too–big data. Bowker (2013) notes that “much of our knowledge today surpasseth human understanding” [13]. Weinberger clarifies that this problem is not one of quantity but of oversimplifying the causal connections between data, information, knowledge and wisdom. Knowledge is not the result of filtering data into information. Rather:

It results from a far more complex process that is social, goal–driven, contextual, and culturally–bound. We get to knowledge — especially “actionable” knowledge — by having desires and curiosity, through plotting and play, by being wrong more often than right, by talking with others and forming social bonds, by applying methods and then backing away from them, by calculation and serendipity, by rationality and intuition, by institutional processes and social roles. [14]

At the very least, identifying more nuanced frameworks for what inquiry means as a process can help augment our knowledge production by forcing us to articulate clearly how the stuff we want to call data is relevant, given the specific questions and goals of our inquiry. More radically, embracing the complexity of inquiry as a generative process of collaborative remix can push us to accept that no matter how good our tools, algorithms, or filters, we cannot possibly explain the whole of any situation. This is a good thing to realize, as it allows us to stop trying and move on to more creative and innovative insights. End of article

 

About the author

Annette N. Markham is Associate Professor of Information Studies at the Department of Aesthetics and Communication at Aarhus University. Her research focuses on qualitative research methods and ethics of Internet–mediated contexts. Her sociological work on lived experience in Internet contexts is well represented in the book Life online: Researching real experience in virtual space (Alta Mira, 1998). Her most recent research focuses on the concept of remix as a way of reframing qualitative inquiry in contexts saturated with social media. In addition to her co–edited volume, Internet inquiry: Conversations about method (Sage, 2009, with Nancy Baym), her work appears in a range of books and peer–reviewed journals. Annette Markham received her Ph.D. in organizational communication in 1997 at Purdue University.
E–mail: amarkham [at] gmail [dot] com

 

Acknowledgments

Thanks to the fellow contributors of this special issue of First Monday for providing valuable advice throughout the construction of this paper. Also, thanks to my Facebook friends for helping me come up with what I hope remains a reasonable title.

 

Notes

1. In this particular paper, I bypass many ancillary questions, such as: “Where do these frameworks come from?” and “How do these frames influence our everyday behaviors as scientists?” and choose instead to focus on what would happen if we identified these invisible frames and chose something different. I don’t mean to gloss these other prior questions, but they are much more adequately addressed by others, over decades of epistemological discussions.

2. All screenshots are taken by the author from publicly accessible areas of the Web. Following the best practice ethical principles outlined by the International Communication Association (2010), we assess that the use of these materials falls well within the U.S. doctrine of ‘fair use.’

3. Geertz, 1973, p. 9.

4. Law and Mol, 2002, p. 11.

5. Weinberger, 2011, p. 119.

6. Markham, 2012, p. 6.

7. E.g., the use of fragmented narrative as a means of distributing power in the academic writing form (Markham, 2005); the idea of inquiry as bricolage (Lévi–Strauss, 1966; Kincheloe, 2001, 2005); the use of metaphors such as dance (Janesick, 2001), jazz (Oldfather and West, 1994), and crystallization (Richardson, 1994; Ellingson, 2009); and, the consistent development of innovative interpretive forms of inquiry among primarily U.S. scholars of what could be called second Chicago school of sociology which emerged in the late 1980s and 1990s.

8. Markham, 2013, pp. 69–70.

9. Weinberger, 2011, p. 156.

10. I borrow this phrase from Bruno Latour, whose work influences this paper. I use this phrase more broadly than his article of the same title (Latour, 2004), using the term as suggested by both his previous (e.g., Latour, 1993) and later (e.g., Latour, et al., 2012) works.

11. Rosenberg, 2013, p. 18.

12. Rosenberg, 2013, p. 18; see also Bowker, 2013, p. 170.

13. Bowker, 2013, p. 169.

14. Weinberger, at http://blogs.hbr.org/cs/2010/02/data_is_to_info_as_info_is_not.html, accessed 10 September 2013.

 

References

Tom Boellstorff, 2012. “Epistemologies of ethnography,” invited presentation at Association of Internet Researchers (AoIR) 13th annual meeting, Manchester, U.K. (21 October).

Geoffrey Bowker, 2013. “Data flakes: An afterword to ‘raw data’ is an oxymoron,” In: Lisa Gitelman (editor). ‘Raw data’ is an oxymoron. Cambridge, Mass.: MIT Press, pp. 167–172.

Geoffrey Bowker, 2005. Memory practices in the sciences. Cambridge, Mass.: MIT Press.

danah boyd and Kate Crawford, 2012. “Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon,” Information, Communication & Society, volume 15, number 5, pp. 662–679.
doi: http://dx.doi.org/10.1080/1369118X.2012.678878, accessed 18 September 2013.

Kenneth Burke, 1966. Language as symbolic action: Essays on life, literature, and method. Berkeley: University of California Press.

James Clifford and George Marcus (editors), 1986. Writing culture: The poetics and politics of ethnography. Berkeley: University of California Press.

Laura Ellingson, 2009. Engaging crystallization in qualitative research: An introduction. Thousand Oaks, Calif.: Sage.

Kirby Ferguson, n.d. “Everything is a remix,” at http://everythingisaremix.info, accessed 14 January 2013.

Harold Garfinkel, 1967. Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.

Clifford Geertz, 1973. The interpretation of cultures: Selected essays. New York: Basic Books.

Lisa Gitelman (editor), 2013. ‘Raw data’ is an oxymoron. Cambridge, Mass.: MIT Press.

Erving Goffman, 1974. Frame analysis: An essay on the organization of experience. New York: Harper & Row.

Beki Grinter, 2013. “A big data confession,” Interactions, volume 20, number 4, pp. 10–11.
doi: http://dx.doi.org/10.1145/2486227.2486231, accessed 18 September 2013.

International Communication Association, 2010. “Code of best practices in fair use for scholarly research in communication,” at http://www.centerforsocialmedia.org/fair-use/related-materials/codes/code-best-practices-fair-use-scholarly-research-communication, accessed 19 September 2013.

Jean Jackson, 1990. “‘Deja entendu’: The liminal qualities of anthropological fieldnotes,” Journal of Contemporary Ethnography, volume 19, number 1, pp. 8–43.
doi: http://dx.doi.org/10.1177/089124190019001002, accessed 18 September 2013.

Valerie Janesick, 1994. “The dance of qualitative research design: Metaphor, methodolatry, and meaning,” In: Norman Denzin and Yvonna Lincoln (editors). Handbook of qualitative research. Thousand Oaks, Calif.: Sage, pp. 209–219.

Joe Kincheloe, 2005. “On to the next level: Continuing the conceptualization of the bricolage,” Qualitative Inquiry, volume 11, number 3, pp. 323–350.
doi: http://dx.doi.org/10.1177/1077800405275056, accessed 18 September 2013.

Joe Kincheloe, 2001. “Describing the bricolage: Conceptualizing a new rigor in qualitative research,” Qualitative Inquiry, volume 7, number 6, pp. 679–692.
doi: http://dx.doi.org/10.1177/1077800405275056, accessed 18 September 2013.

George Lakoff and Mark Johnson, 1980. Metaphors we live by. Chicago: University of Chicago Press.

Bruno Latour, 2004. “Why has critique run out of steam? From matters of fact to matters of concern,” Critical Inquiry, volume 30, number 2, pp. 225–248.
doi: http://dx.doi.org/10.1086/421123, accessed 18 September 2013.

Bruno Latour, 1999. Pandora’s hope: Essays on the reality of science studies. Cambridge, Mass.: Harvard University Press.

Bruno Latour, 1993. We have never been modern. Translated by Catherine Porter. Cambridge, Mass.: Harvard University Press.

Bruno Latour, Pablo Jensen, Tommaso Venturini, Sébastian Grauwin, and Dominique Boullier, 2012. “‘The whole is always smaller than its parts’ — A digital test of Gabriel Tardes’ monads,” British Journal of Sociology, volume 63, number 4, pp. 590–615.
doi: http://dx.doi.org/10.1111/j.1468-4446.2012.01428.x, accessed 18 September 2013.

John Law and Anemarie Mol (editors), 2002. Complexities: Social studies of knowledge practices. Durham, N.C.: Duke University Press.

Lawrence Lessig, 2008. Remix: Making art and commerce thrive in the hybrid economy. New York: Penguin Press.

Claude Lévi–Strauss, 1966. The savage mind. Chicago: University of Chicago Press.

“McArthur universal corrective map of the world,” at http://odtmaps.com/detail.asp_Q_product_id_E_McA-23x35, accessed 1 September 2013.

Annette Markham, 2013. “Remix culture, remix methods: Rethinking qualitative inquiry for social media contexts,” In: Norman Denzin and Michael Giardina (editors). Global dimensions of qualitative inquiry. Walnut Creek, Calif.: Left Coast Press, pp. 63–81.

Annette Markham, 2012. “Fabrication as ethical practice: Qualitative inquiry in ambiguous Internet contexts,” Information, Communication & Society, volume 15, number 3, pp. 334–353.
doi: http://dx.doi.org/10.1080/1369118X.2011.641993, accessed 18 September 2013.

Annette Markham, 2005. “‘Go ugly early’: Fragmented narrative and bricolage as qualitative method,” Qualitative Inquiry, volume 11, number 6, pp. 813–839.
doi: http://dx.doi.org/10.1177/1077800405280662, accessed 18 September 2013.

Annette Markham and Simon Lindgren, in press. “From object to flow: Network sensibilities, symbolic interactionism, and social media,” Studies in Symbolic Interactionism, pre–publication draft at http://www.markham.internetinquiry.org/writing/MarkhamLindgrenOptimizedForBlog.pdf, accessed 18 September 2013.

Nicholas Negroponte, 1995. Being digital. New York: Knopf.

Penny Oldfather and Jane West, 1994. “Qualitative research as jazz,” Educational Researcher, volume 23, number 8, pp. 22–26.
doi: http://dx.doi.org/10.3102/0013189X023008022, accessed 18 September 2013.

“Peters projection world map,” at http://www.petersmap.com/table.html, accessed 5 September 2013.

Sarah Pink, 2012. Situating everyday life: Practices and places. Thousand Oaks, Calif.: Sage.

Michael Reddy, 1979. “The conduit metaphor: A case of frame conflict in our language about language,” In: Andrew Ortony (editor). Metaphor and thought. Cambridge: Cambridge University Press, pp. 284–310.

Laurel Richardson, 1994. “Writing: A form of inquiry,” In: Norman Denzin and Yvonna Lincoln (editors). Handbook of qualitative research. Thousand Oaks, Calif.: Sage, pp. 516–529.

Daniel Rosenberg, 2013. “Data before the fact,” In: Lisa Gitelman (editor). ‘Raw data’ is an oxymoron. Cambridge, Mass.: MIT Press, pp. 15–40.

David Weinberger, 2011. Too big to know: Rethinking knowledge now that the facts aren’t the facts, experts are everywhere, and the smartest person in the room is the room. New York: Basic Books.

David Weinberger, 2010. “The problem with the data–information–knowledge–wisdom hierarchy,” Harvard Business Review Blog Network (2 February), at http://blogs.hbr.org/cs/2010/02/data_is_to_info_as_info_is_not.html, accessed 10 September 2013.

 


Editorial history

Received 16 September 2013; accepted 17 September 2013.


Creative Commons License
This paper is licensed under a Creative Commons Attribution–NoDerivs 3.0 United States License.

Undermining ‘data’: A critical examination of a core term in scientific inquiry
by Annette N. Markham.
First Monday, Volume 18, Number 10 - 7 October 2013
http://firstmonday.org/ojs/index.php/fm/article/view/4868/3749
doi:10.5210/fm.v18i10.4868.





A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2017. ISSN 1396-0466.