First Monday

Good for tech: Disability expertise and labor in China's artificial intelligence sector by Di Wu



Abstract
People with disabilities are often perceived as being “given” the opportunity to work, rather than “providing” valuable labor. Centering on disabled data workers as experts involved in the quotidian construction of artificial intelligence (AI) systems in China, this article shows that disability expertise and labor can afford a technical edge to AI systems in a certain political economy. In the case examined, the work of consistently synchronizing interpretations of the ambiguous data and elusive rules of smart home systems prefers a stable annotation workforce with coordinated cognition and trained judgment. This technical demand has come to be met by a committed team of skilled disabled workers, who are pushed out from mainstream job market by systemic ableism, and pulled in by disability-informed expertise that reconfigures space, time, and political economy to meet non-normative bodyminds. Through this exceptional case run by a disabled people led organization, I draw attention to disabled people’s under-examined role as system-builders of information technologies as opposed to users, victims, or inspirations, and highlight the transformative potential of disability expertise.

Contents

Introduction: Tech for good, or good for tech?
Objectified resource, undervalued expertise
Stability as technical edge
Escaping ableism: “I’d do anything but massage!”
Embracing disability expertise: “Ghost work” alternatives
Conclusion

 


 

Introduction: Tech for good, or good for tech?

“Microwork Helps Disabled People Fly Against the Light.”
“AI Brings New Jobs for Disabled People’s Employment.”
“Big Data Annotation Heralds New Disability Assistance Model.”

Numerous news headlines in China have proclaimed people with disabilities to be the beneficiaries of new kinds of job opportunities serving the development of artificial intelligence (AI) systems (e.g., Netease.com, 2018), such as data annotation. Essential for machine learning systems to recognize patterns from a dataset, data annotation often entails a large number of human workers to hand code, label, and sort training data. In recent years, government agencies and technology companies in China have enthusiastically set up programs aimed at recruiting and training disabled people to conduct data annotation work, often branded “tech for good.” Such jobs, they claim, empower disabled people by increasing their income, by offering ways to overcome physical barriers, and by making them “useful” to society thus fulfilling these workers’ “self-worth” (JD.com, 2018).

Scholars on technology labor have demystified such narratives. Data annotation work, often organized in the form of microwork or crowdwork, is known for exploitative labor conditions that underpay, deskill, and surveil workers, who must stay hypervigilant to compete for tasks, with little means for collective action (Irani, 2016, 2015; Ekbia and Nardi, 2017; Gray and Suri, 2019; Roberts, 2019; Altenried, 2020; Vallas and Schor, 2020). Disability studies scholars have also detailed how digital work may simultaneously empower and exploit, include and exclude, people with disabilities (Yu, et al., 2019). On the one hand, information technologies have provided affordances for people with certain disabilities to bypass discrimination (Gray and Suri, 2019), to work around physical inaccessibility (Dobransky and Hargittai, 2006), to forge communities of resistance and self-governance (Lin and Yang, 2020), and to express agency, self-identity (Goggin, et al., 2019), and a sense of contribution (Boellstorff, 2019). On the other hand, new forms of algorithmic cruelty (Irani, 2016; Gray and Suri, 2019), digital inaccessibility (Zyskowski, et al., 2015), trade-offs between flexibility and security (Qu, 2022), and exacerbated precarity (Lin, et al., 2019) abound in information technology work for people with disabilities. Some disability theorists question the capitalist imperative to be productive altogether (Mitchell and Snyder, 2010).

This paper moves beyond assessing whether tech is truly “for good.” Rather than debating how tech work may empower or exploit people with disabilities, I focus on how disabled people’s labor may in turn transform technology. I show that the labor and expertise of disabled people can in fact be “good for tech.” The disabled workers examined in this study provide highly competitive data annotation services to their big tech client, “not by playing the charity card or fighting a price war,” in their own words, but by fulfilling a key structural need of the AI system. I approach people with disabilities as not the receiving end of technological promises and perils, but as experts involved in the quotidian construction of “intelligent” systems. This is inspired by information studies’ attention to the materiality of technical configurations, as well as disability studies’ emphasis on the agency of disabled people. Taken together, the two fields shed new light on the intricate relationship between disability and information technology.

My analysis focuses on one data annotation program run by a disabled persons’ organization (DPO) that I call ENABLE. I draw upon 19 remote interviews and two field trips between early 2020 and summer 2021, as well as my own five-year professional experience of managing disability advocacy programs in China’s non-profit sector. My interlocutors are a team of workers with visual or physical impairments, hired to label Chinese natural language processing (NLP) data for smart home technologies developed by a leading Chinese tech company I call AITech. ENABLE recently outperformed many non-disabled competitors and became a major data annotation service provider for AITech. This case is rather an exception, not the norm, of data annotation programs in China. As I will elaborate, run by disabled people themselves, the examined program has distinctive approaches to data labor.

Centering on the embodied experience of disabled workers and the inner workings of sociotechnical processes, I argue that the labor and expertise of disabled workers offer the AI system a technical edge. My argument is three-fold. First, as ENABLE’s workers taught me, data annotation for smart home AI systems entails human interpretations of highly ambiguous data, to which the rules can iterate on a weekly basis. To consistently formalize tacit knowledge that improves the system’s response to user queries, a stable workforce of human annotators with coordinated cognition and trained judgment is preferred. Second, this technical demand has come to be met by a committed team of skilled disabled workers, who are pushed out from the mainstream job market by structural ableism. Third, and more importantly, the workers are pulled in by the DPO’s unorthodox configurations of workplace to meet the workers’ heterogenous bodyminds. I use anthropologist Cassandra Hartblay’s concept of “disability expertise” to unpack the disability-informed knowledge of flourishing in uninhabitable worlds (Hartblay, 2020). In addition to non-normative spatial and temporal strategies, the DPO deployed a form of disability expertise that I call “resource hacking,” optimizing precarious resources in ableist political economies for maximum disability gain, or in one interlocutor’s words, “making resources serve you even when you don’t have any.” Disabled workers offer what anthropologist Michele Friedner calls “multiple regimes of value” to techno-capitalism, not only through reinscribing stigma (Friedner, 2015b), but also the deployment of genuine expertise acquired through the lived experience of disability. Through this case, I bring to the fore the role of disabled people as system-builders of information technologies rather than users, victims, or inspirations, and highlight the transformative potential of disability expertise.

 

++++++++++

Objectified resource, undervalued expertise

Disability has routinely figured as an epistemic and material resource in the history of information technology (Wu, 2021). The sound spectrograph, a precursor to speech recognition technologies (Li and Mills, 2019), was initially proposed to improve deaf education (Mills, 2010). Early time-compression technologies were popularized by blind aural speed-readers (Sterne and Mills, 2020). At robotics and AI labs at MIT, analogies between disabled bodies and machines were a recurring tool to theorize disembodied intelligence (Richardson, 2015). Disability frequently serves as a metaphor, precursor, or advertisement for the research, production, and commercialization of technoscientific objects that are ultimately designed for non-disabled users, a pattern that historian Mara Mills (2010) calls “assistive pretext.” Disability is also constantly invoked as a “narrative prosthesis” (Mitchell and Snyder, 2001) to metaphorize arguments in cultural studies of technoscience, in over-objectified and under-constructed ways (Jain, 1999). In contemporary urban India, anthropologist Michele Friedner shows that the immobility, sociality, and affect of deaf workers added multiple forms of value to their employers, many in the information technology sector, in ways that do not always benefit deaf people (Friedner, 2015b).

While the technoscientific extraction of value from objectified disabled body-minds proceeds apace, the actual knowledge and expertise of disabled people are ironically ignored. Science and technology studies scholar Ashley Shew (2020b) laments that disabled people are too often enrolled as “marginal cases” and objects of “thought experiments” in technoscientific imaginaries, while being subject to “epistemic violence” (Ymous, et al., 2020) that denies them a role as real experts and legitimate knowers. Disability scholars Aimi Hamraie and Kelly Fritsch (2019) urge us to center disabled people as experts in technoscience, and as agents who can harness technology for political change, proclaiming what they call a “crip technoscience.” Likewise, anthropologist Cassandra Hartblay calls attention to “disability expertise,” forms of knowledge that “disabled people develop about unorthodox configurations of agency, cultural norms, and relationships between selves, bodyminds, and the designed world.” [1]

This article brings the critique of value extraction and the recognition of disabled people’s expertise into conversation. In the case examined, legitimate disability expertise — in tension and in tandem with objectified resourcing of disability premised on stigma and inequality — forms a crucial part of the value added by the disabled workers. I show that when crip technoscience and disability expertise are practiced, the value of disability need not be created entirely at the expense of disabled people, and labor relations in information technology can be otherwise.

 

++++++++++

Stability as technical edge

Data annotation for smart home AI systems is far more complicated than rote “click work.” To excel at such work takes expertise. As of 2020, ENABLE has outperformed many non-disabled competitors, becoming a major supplier of data annotation services for AITech. In my experience as a non-profit professional, most corporations’ motivation to hire disabled people was to fulfill a disability employment quota mandated by the Chinese government, because non-compliance could result in a significant financial penalty (Liao, 2021). Often, even such strong incentives could not persuade employers. Many would rather pay the fine, or rent a disability certificate as a token of compliance without giving the disabled person real work (Qu, 2020). Surprisingly, at the time of this research, AITech has not claimed any disability employment quota from the hiring of ENABLE’s disabled workers. Neither, as is often the case, are the disabled workers receiving less pay for the same work as non-disabled annotators.

To understand what makes ENABLE’s labor so competitive, we need to unpack what the job entails. Tasked with classifying user intentions, rating sound qualities, or sometimes determining the gender and age of smart device users, the annotators engage in complex decision-making processes that demand skilled listening and selective in/attention (Semel, 2022). I argue that the act of classifying human intentions and interpreting sounds amidst constant iterations of commercial AI algorithms, is intrinsically ambiguous and volatile. This technical condition makes a committed data annotation workforce with trained judgment and coordinated cognition more productive than forms of labor that are highly flexible and contingent.

Ambiguous data

Contemporary voice-based AI systems built on supervised machine learning algorithms require huge data sets of high-quality, annotated user requests (Tubaro, et al., 2020). The blind and low vision workers’ official job title is “intent annotator.” Every day, they listen and determine the intention of user queries to voice-activated smart home systems, and code them into thousands of specific “features” that trigger the correct machine response. For example, if a user query reads “turn on the AC” in Chinese, annotators will sort it into “hardware control.” It sounds straightforward. But not all queries are this clear-cut. For instance, a surprisingly large number of users engage these devices in casual conversations. In this case, annotators need to determine what emotions are conveyed, whether the user is talking with another human or interacting with the device, making a command, or simply saying nonsensical words. Sometimes, the name of a song could sound like a conversation and escape an ear unattuned to the latest trends in pop culture. Lihua, a well-educated blind annotator, often finds herself frustrated by the ambiguity of the data. She describes it as making meaning out of “a broken conversation.”

The conversation is “broken” for many reasons. In between the annotator and the user lie multiple layers of cuts and transcriptions. The user’s speech is first cut out of their social context, converted from analog audio information to digital data, then transcribed from speech to text, and segmented into short phrases for annotation. Annotators, without necessarily hearing the human voice or the full sentence of the query, now need to judge the user’s “true” intention. Here is an example given by an annotator. In Mandarin Chinese, the written form of “you say you like me huh” (ni shuo ni xi huan wo ma) can at least allow three interpretations: “Did you say that you like me?” “Say that you like me!” or “Do you think you like me?” Without knowing the intonation and punctuation from the original speech, it is difficult to judge the precise meaning or emotions.

Blind annotators use screen readers — software applications that convert text into synthetic speech — to read the textual data. Screen readers are key access tools for the blind and low vision workers. Through ENABLE’s negotiation, AITech made their annotation system screen-reader compatible, so workers can listen to the content of the data and navigate the portal aurally. One may assume that annotating with a screen reader would be a disadvantage for blind workers. But sighted annotators’ experience suggests that the confusion was less caused by sensory differences than by the data’s lack of social and linguistic context. Kai is a sighted annotator with physical impairments, who works on the same types of data as the blind annotators. To him, reading those texts with vision is just as “brain burning.”

The act of classification is intrinsically reductive (Bowker and Star, 1999). User intent classification attempts to impose an artificial social order (Suchman, 1993) onto the messy, complex inner world of human users so their needs can be made legible to the machine. Data annotators are part of the layers of mediation that render the technology and end users mutually legible and constructive to each other (Robbins, et al., 2021). Far from an impartial, objective, and rote act of simple “click work,” data annotation is a “sense-making process” (Klein, et al., 2007), in which human workers with heterogenous lived experience assign meaning to snapshots of decontextualized content with prescribed labels. To excel at such work means producing consistent interpretations for not only the ocean of “commonsensical” data, but also the “brain burning” edge cases. Consistency comes from experience and coordination.

Elusive rules

If reading minds in broken words is challenging, judging sound in isolation can be equally frustrating. Staffed by predominantly sighted wheelchair users, ENABLE’s team in western China focuses on sorting sound clips, such as identifying the “wake word” (for example, “Alexa” or “Hey Google” are wake words for their respective devices), determining the age or gender of the speaker, telling speech from non-speech, and rating the clarity of speech. These sound clips are sent to the annotators for manual identification, precisely because they were accented, unclear, or confusing. Their job is to literally separate signal, or the lack thereof, from noise.

Though meticulous rules for how to annotate exist, interpretation of the rules is arbitrary. Wenbo is a young man with a humorous and relaxed demeanor. Sitting in his wheelchair, he had a cup of green tea and a cigarette on the table while we video called. The moment I asked him about annotation standards, he suddenly got serious:

Speaking of this, let me just say — there is no standard! If a lighter costs fifty cents, it costs fifty cents. But for things like sound, everyone’s ears are different, and everyone’s accents are different. [...] If the wake word was spoken very fast, I may find it OK and clear. But if the Quality Assurance (QA) person finds it unclear, then it’s not OK.

The ambiguity of judging sound is a recurring theme among workers who sort audio data. Meihui has a college degree and uses a wheelchair. For her, the hardest part of the job is to listen “objectively” and mechanically to something “subjective” and animate. In her view, “sound is meant to be a living thing! [...] But the QA would apply dead rules to judge our work.” Multiple annotators were frustrated when they listened to the same thing as the QA person but heard differently. The QA has the power to determine the “accuracy rate” of the annotated data by spot-checking datasets. If the set has an accuracy rate lower than 98 percent, the entire set would be sent back for the annotators to rework.

The heterogeneous, situational, and immersive qualities of sound (Sterne, 2003; Helmreich, 2007; Zdenek, 2015) frequently clashed with the rigid corporate quest for a single absolute meaning of an isolated sound bite, arbitrarily ruled by ears with higher epistemic authority. To mediate such tension, the workers’ strategy is to train themselves to listen like the QA (who supposedly represents the client) irrespective of what they actually hear. As human-computer interaction scholars argue in the context of computer vision data annotation, power dynamics, and organizational hierarchy have a more profound impact on the outcome of data annotation than individual bias (Miceli, et al., 2020).

Most challengingly, the constant iterations of commercial AI algorithms demand frequent updates in the annotation rules. The annotators recall modifications in rules on a weekly basis. Each time a new specification, combination, or option becomes available, the calculation of the “correct” label is subject to change. Not only are annotators required to learn the rules promptly, but the more knowledge they have about the genealogy of the existing rules, the quicker they can preclude wrong interpretations of the new rule. In other words, annotators with stronger institutional memory can better recognize patterns in the ever-changing rules, and make more accurate predictions of interpretation.

The disabled workers’ long-term knowledge also brings direct improvements to AITech’s product. To synchronize data interpretations between the annotators, QAs, and the client, the disabled workers attend weekly meetings with the developers. In these meetings, the annotators provide direct feedback on trends, problems, and recommendations. Often, the developers end up adjusting the feature of the product based on common queries that the annotators observe. As Danni, a geeky, young blind woman, remarks, “we are the ones who understand the users most.” The close feedback loop ensures that developers know what is happening on the ground.

The disabled workers function as skilled technicians who glue together elusive tacit knowledge derived from shifting managerial and technical specifications in a fast-paced institution. In the face of endless new data and new rules, high-quality data annotation means consistently predicting the preferred interpretations by the client and QAs. The best way for the annotators to hone such skills is through long-term, close collaboration, with iterative feedback from QAs, developers, and users. ENABLE’s director Zhang Fan takes pride in the fact that their annotators “get” AITech developers better, because they have been around longer, and are more familiar with earlier versions of the annotation rules, than the non-disabled QAs who have a much higher turnover rate.

Data annotation tasks and labor forms are not homogenous. While some tasks can be satisfactorily performed by part-time, flexible crowdworkers managed by platform algorithms, or short-term annotation “factories,” other tasks may favor a full-time, trained workforce that is in close coordination with developers (Lavee, et al., 2019; Semel, 2022). Forms of labor are contingent upon the nature of the data and tasks, level of secrecy, project duration, layers of outsourcing, and localized political economy of annotation labor. In the case of AITech, to synchronize the interpretations of ambiguous data and elusive rules, a constant workforce of data annotators who have rich tacit knowledge, good institutional memory, and a strong working relationship with the developers, stands out as superior to other more flexible workers. The quality of the data is thereby closely tied to the stability of the annotation workforce. Here, stability is not achieved through platform-based labor that is always-on, spatially heterogenous, and temporally flexible (Altenried, 2020), but by capitalizing upon structural ableism and disability expertise.

 

++++++++++

Escaping ableism: “I’d do anything but massage!”

The stability of disabled workers has come to be instrumental to the quality of data annotation for AITech. As ENABLE’s workers note, it is not easy to find so many experienced annotators, and most non-disabled workers would not stay in the job as long because it is “too tedious.” However, disabled workers are not naturally stable but are rendered so. In this section, I argue that the disabled workers, pushed out of mainstream labor market and pigeonholed into a few job options by structural ableism, long for a sense of mobility that, ironically, hinges upon supplying their immobility to the AI company.

The labor regime of disability in contemporary China is strongly shaped by state biopolitics and unchecked ableism. Canji, the legal category of disability, defines disability based on biomedically measured deficits in bodies and minds (Kohrman, 2005). This medicalized ideology imposes rehabilitation as the pre-requisite for disabled people’s full participation in society, deprioritizing accessible infrastructures (Cui, et al., 2019), equal education (Hu and Lin, 2017), and competitive work opportunities. Although official data estimates that 56 percent of working-age disabled people are employed (Chinanews.com, 2019), nearly half of these jobs are categorized as agriculture and cultivation, and over another quarter are “flexible” employment (China Disabled Persons’ Federation, 2021), namely temporary or part-time jobs. Key targets of poverty alleviation campaigns, households of disabled people earned only 57.1 percent of the national average income in 2018 and the gap continues to widen (Cheng, 2021).

Since the market reforms, the state has attempted to leverage economic incentives for private employers and self-employment, in contrast to the direct state investment in welfare enterprises during the Mao era (Huang, et al., 2009). The employment quota policy is a major instrument. Instituted in the 1990s, the policy mandates public and private entities to reserve at least 1.5 percent of their workforce for people with disabilities. Failure to comply shall result in financial penalty. In practice, however, the system is poorly enforced, and has created perverse incentives (Liao, 2021), such as performing “fake employment” that rents a disabled person’s certificate without real work.

Coupled with the lack of systemic opportunities is the abundance of restrictions. For many blind and low vision workers at ENABLE, their life trajectory was prescribed early on — go to special schools, take separate exams, and, if lucky, go to a special education college to study massage and become a massage therapist. Introduced as vocational education programs for students in blind schools in the 1930s (Tie, et al., 2011), massage programs have gradually morphed into an institutionalized effort to generate scalable jobs for blind people in contemporary China (Li, et al., 2022). The protective policies had constraining consequences. Massage has since been culturally imagined as the default occupation for people with visual impairments, and blind people as incapable of other professions (Dauncey, 2020; Li, et al., 2022). As a result, while mainstream universities remain inaccessible to most blind students (Hu, 2022), in special higher education, few majors are open to blind students. Roughly 80 percent of visually impaired students in special colleges major in massage (Li and Fu, 2015), many of ENABLE’s workers included.

“Escaping from massage” becomes a key drive for ENABLE’s blind and low vision workers to explore data annotation as a new profession. Some blind annotators lamented that they would do “anything but massage.” Similarly, when asked about their interest in data annotation, sighted workers with physical impairments frequently speak about the desire to “leave home,” “go out,” and “experience life,” instead of interest in the work itself. At play is less of a “politics of destination” where immobility is experienced as displacement (Chu, 2010), but a “politics of escape” in which departure from an assigned destination is an end itself. With a few exceptions, most workers at ENABLE came from an urban, lower to middle class background. Compared to China’s predominantly rural disabled population, they had relatively more means to defy their “destiny” and endure the precarity of constant circulation between gigs, trainings, and jobs. Data annotation emerged in the array around 2016.

Paradoxically, the workers’ yearning for mobility has come to rely on marketing their “stability.” Workers often cite “stability” as their comparative advantage to their non-disabled competitors. Probed further, however, they would speak of “lack of better options,” or “taking jobs more seriously.” To them, “stability” is essentially a euphemism for social and physical immobility. As Friedner (2015a, 2015b, 2014) has argued, the stigma of disabled people’s immobility is often reinscribed as value in the name of stability in late capitalism. Despite constant motion of moving around horizontally, Chinese disabled workers feel that they barely move upward in the vertical social ladder.

By contrast, tech discourse in China often naturalizes such stability. In a commentary by a well-subscribed tech media, people with visual impairments are described as “naturally gifted” to do user intent annotation, because they are “used to listening to information” and have “very strong meaning comprehension and linear logical abilities” which makes them “understand and reconstruct meaning better than normal people.” Disability hereby is essentialized as a fixed biological reality, rather than a relational and political experience (Kafer, 2013). In addition, the article notes, “with fewer visual interferences, their attention is more focused.” Although reframing a commonly assumed deficiency as an advantage may seem uplifting, the theory is rebutted by the workers for downplaying the structural restrictions imposed upon blind people. Like the indigenous women portrayed as “natural” circuit assembly workers in 1960s United States (Nakamura, 2014), disabled women and men in contemporary China become naturalized labor for AI companies. Similar discourses can be found today in the United States, where autistic data workers are often depicted as technologically gifted for AI but denied personhood (Keyes, 2020). At ENABLE, some annotators themselves may resort to naturalizing narratives when making a case about their productivity; but almost all of them are wary of attempts to pigeon-hole them yet again. As Yang, a shy blind man remarks, “if all blind people start doing data annotation, then it is like massage all over again!”

Unfortunately, naturalizing the stability of disabled workers has material consequences. Despite their evident value and proven skills, ENABLE annotators have limited prospects for upward mobility within AITech. Chunlin is an experienced blind annotator. When we talked in 2020, she had worked for over two years as an annotator, but witnessed AITech’s non-disabled annotators being promoted in less than a year:

Chunlin: Regardless of how good you are at annotation, your opportunities are frustratingly limited.
Me: Why?
Chunlin: Many reasons. From my perspective, I can see that accessibility is an issue. Currently AITech only made the annotation portal accessible, but not the portal for arbitrators [a higher-level position than annotators].
Me: Why didn’t they modify the arbitrators’ portal?
Chunlin: My guess is that they never thought about making us arbitrators.

Indeed, if annotation is where disabled people “naturally” belong, then there is no need to plan for a path that involves them. The opposite of accessibility is therefore not inaccessibility, but restricted access (Ellcessor, 2016), preserved for people who are deemed more mobile and worthy.

Disabled data annotators at ENABLE provided a crucial human resource to enhance the quality of smart home AI data annotation. The absence of better job opportunities due to systemic ableism and the medicalization of disability in China explains what pushed many ENABLE workers out of the mainstream job market. In the next section, I unravel what pulls many workers to stay despite limited career development.

 

++++++++++

Embracing disability expertise: “Ghost work” alternatives

Limited opportunities aside, data annotation is not automatically a superior job option. Multiple ENABLE workers have previously conducted platform-based, home-bound annotation work, and suffered from the same kinds of algorithmic and human cruelty often dubbed as “ghost work” (Gray and Suri, 2019). I argue that it is the disability expertise of ENABLE and their workers that made “ghost work” more humane, pulling many workers to stay in this particular workplace. Disability expertise refers to disability-informed, non-normative knowledge of inhabiting the world. Domains of disability expertise include strategies of managing perception, claiming citizenship, and living under domination (Hartblay, 2020).

Here, I show how disability expertise unfolds in high-tech workplaces. I elaborate on three domains of expertise deployed by the disabled workers that reconfigure space, time, and political economies to serve bodyminds with disabilities. Particularly, I propose a new form of disability expertise that I call “resource hacking,” a set of pragmatic skills that disabled activists, especially in resource-poor contexts, develop through navigating different variants of ableism in government, corporations, and the society, while leveraging their fragmented resources to move the needle for disability advocacy.

Co-creation of access

ENABLE’s operational model stands out from other disability AI data annotation programs — platform-based or non-disabled people led business process outsourcing — in their deliberate efforts of community building. To this end, a shared physical space is crucial. ENABLE devoted tremendous efforts in locating, partnering, and mobilizing funding for accessible and affordable offices and living spaces in major cities. Accommodation is not an afterthought, but a precondition. ENABLE had to reject many clients to avoid compromises on accessibility. Free dormitory space in big cities and a stable salary made it possible for many workers to experience life in different cities with “financial and psychological independence,” as one worker puts it. Many of them became friends through work and organized weekend trips together. Rongfei came from a small village in central China. Growing up, she never met anybody who used a wheelchair like herself. The relentless staring at her wheelchair by people in the street used to discourage her from going out. But now traveling with a dozen wheelchair users together brings her a sense of pride. Meihui, who used to walk with crutches to appear less “crippled,” also echoed how she was liberated by, rather than “confined to,” her wheelchair, thanks to a community of wheelchair users who taught her how to roll. These physical spaces that are made accessible allow ENABLE’s workers a sense of “membership and mattering” (Lynch, 2013).

The meaning of physical space can also be symbolic. ENABLE’s Shanghai office is a typical, white-collar office with about 30 individual cubicles and computers, located in a high-tech compound. Some annotators have posted photos of their office on social media, or have invited parents to visit the office, to show that they now live “a decent life.” Many workers recognize that this does not fit the “inclusive employment” canon as promoted by the United Nations’ Convention on the Rights of Persons with Disabilities (CRPD), in that they are not working alongside non-disabled colleagues. Nevertheless, annotators feel more socially integrated by doing ordinary things like commuting, working for eight hours a day, or even squabbling with discriminatory neighbors. A few disabled women especially highlighted to me how they appreciated working “with a computer” and inhabiting an office that looks “white-collar” to their family and friends. The physical workspace and particular appearance of informatics work can be integral to workers’ job experiences and even their identities (Freeman, 2000). These embodied, physical “disability worlds” (Ginsburg and Rapp, 2013) made the work much more meaningful for the annotators than just clicking alone on a computer at home earning the same income.

Digital spaces are also made accessible, a feature essential for the blind and low vision workers to annotate texts. According to director Zhang Fan, who himself is a man with low vision and uses magnifiers and screen readers, the negotiation with the client to advocate for screen-reader compatibility is a delicate process: “If you ask for too much, the client freaks out and finds you troublesome. But if you ask for too little, then the system is not usable.” It requires constant “frictioned negotiations of access and privilege” [2]. One advocacy success, in ENABLE’s view, is that after working together and witnessing the productivity of blind people, AITech developers started proactively consulting ENABLE for accessibility advice before significant system updates.

Even if the portal is made screen-readable, effective access still requires situated knowledge of diverse epistemologies. All visually impaired people do not work alike. As education researcher Lucia Hasty points out, visual learners process information from “whole to part,” whereas non-visual learners may approach from “part to whole” (Hasty, n.d.). Each of these categories of epistemology are infinite spectrums. For example, screen readers typically read a Web page from top-to-down, left-to-right. Blind annotators find it inefficient. Jiabao is one of the programming enthusiasts in the office. In his words, “screen reader compatible systems are merely functional, but not efficient. Efficient systems should create a near-non-disabled experience.” I interpreted his words as not a desire to become normates, but the demand for considering blind people’s epistemic approach as the starting point of access, rather than rigidly translating one sensory modality into another (Lundgard, et al., 2019). This is the job of Wenyu, ENABLE’s in-house blind programmer, who develops shortcuts and add-ons so annotators can “fold” the linear reading sequence of screen readers using keyboards. Meanwhile, low vision workers like Shujun do not necessarily use screen readers. Her access needs are an appropriately sized monitor and a laptop stand that allows her to lean her face towards the screen. Because she sees shapes and colors, Shujun often jumps to where she wants to click based on her memory of the shape of the text.

Access is also profoundly relational at ENABLE. Sighted and blind workers are paired to work on the same data set, so they can compare results and ensure a higher success rate. Off work, blind workers put their hands on the shoulders of low vision workers and walk to the subway station together. While travelling, those who use crutches and those with wheelchairs assist each other in different tasks. The disabled workers are relying not on high-tech solutions, but rather on what anthropologist Arseli Dokumaci calls “microactivist affordances,” namely, everyday acts through which disabled people become affordances for one another in the absence of a readily accessible environment (Dokumaci, 2020). Tacit techniques such as left-and-right hand coordination, or monitor settings that do not hurt eyes, are circulated among workers and staff members through collective trial and error. Access at ENABLE is not a standardized checklist, but a never-ending process of “care work” (Bennett, et al., 2020) centered on the ethics of interdependence (Mingus, 2010).

Crip time at work

The second important practice that made ENABLE a productive workforce for AITech, and a relatively satisfactory workplace to the workers, is their non-normative management of time, namely, “crip time” at work. “Crip time” is a concept and practice deployed by disability activists and scholars to complicate the temporal norms set by industrial time (Hendren, 2020). As disability scholar Alison Kafer puts it, “crip time bends the clock to meet disabled bodies and minds” [3]. Crip time is often imagined as incompatible with work. Words invoked to describe the presumed lack of productivity of disabled persons, often implicate time — “inefficient,” “slow,” “late,” “chronic illness,” or “cannot handle long hours.” ENABLE’s workers recall constant struggles against these temporal stereotypes. Disability scholars and activists, meanwhile, use the notion of crip time to urge a reimagination of human worth that is not bound by economic time (Kafer, 2021).

Workers at ENABLE sought to reconcile the tension between crip time and work within the bounds of corporate production. Instead of segmenting linear, progressive schedules and maximizing unit time productivity, ENABLE created a separate timeline in parallel to, but also intercalated with, corporate time. They negotiated a contract with fixed salaries for the workers specifically to set weekly group-based performance indicators rather than indicators based on individual piecework. Unlike platform-based crowdworkers, who must remain hypervigilant to compete for tasks, the disabled annotators as a whole become a production team. Because labor time does not unfold evenly across the team, individual crip time is respected.

The customization of pace did not necessarily compromise their performance. Delivering in teams guarantees that workers meet the clients’ targets while no one must work overtime. Nevertheless, the fluidity of crip time management can run into conflict with disciplining corporate time, especially with the prevalence of corporate surveillance technologies. For instance, when some workers needed a longer bathroom break to get around in wheelchairs, the QA questioned why they were not detected as “active” in the system for over 30 minutes. Overall, however, through protecting the boundaries of collective crip time, most ENABLE workers with whom I spoke experienced their work as reasonably paced.

Crip time is often perceived as slower than industrial time. But it can also get ahead. To gain control over speed, the annotators developed unconventional listening strategies. For example, if speeded up, screen readers can afford annotators with “speeds that appeared fast to the normate, while feeling timely to disabled people” (Sterne and Mills, 2020). However, most blind annotators find high-speed reading challenging in the context of annotation, because the data itself is already ambiguous and confusing. What they do instead is to disrupt time. As Danni explains:

At first, you may listen to every line in the dropdown menu in order. Once you become familiar, you can directly cut in. You may only listen to a single word of that line and move on. [...] You can just feel that the correct label is in this line.

Rhythms of work become increasingly aligned with the skilled ear. By shortcutting time, the annotators disrupt the imaginaries of linear, progressive time that deem their ways of knowing as necessarily inefficient.

Crip time also encourages budgeting significant time buffers, planning for uncertainties, and refusing to adhere to oppressive timeframes. As Katzman, et al. [4] put it, “crip time reflects the unpredictable, at times defiant, nature of human body-minds.” Contingencies are a built-in feature of crip temporality. ENABLE’s management is aware that the next iteration of AITech’s system may demand more vision in completing certain tasks, such as reading multiple rounds of conversations or labeling underlined content. This is hardly any surprise to the disabled workers, for whom living with uncertainties and exclusion has been the norm. They are prepared for hard negotiations, and more importantly, reserving the option of refusal. Zhang Fan’s plan for when that day comes is to “first, advocate for accessibility. If that’s not possible, then we will negotiate to see if it is ok for us to slow down a little.” Switching clients will be their last resort, but they remain firmly against short-term contracts.

As the COVID-19 pandemic reshuffles scheduling, pace, and the experience of time, many disability scholars argue that we now are all living in crip time (Samuels and Freeman, 2021). Out of sheer necessity, we are finally allowed to be “asynchronous,” to take the time, and to be together in time in ways for which the disability community has long been advocating (Goggin and Ellis, 2020). ENABLE’s practice demonstrates how crip temporal strategies can work in a corporate setting, to bargain with regimes of time where simply no human is fast enough.

Resource hacking

Finally, ENABLE’s expertise of consolidating fragmented resources in ableist political economies for maximum disability gain, or what I call “resource hacking,” is what made the collaboration with AITech possible in the first place. The term is inspired by ENABLE’s director Zhang Fan, who describes their approach as “making resources serve you even when you don’t have any.” We have seen how the AI company profits from the skilled labor, the naturalized immobility, and the collective expertise of disabled workers. But these benefits were not self-evident to many corporations. Rather, it has been through ENABLE’s proactive construction of value that disabled people’s previously denied access is now credited as a “tech for good” success story.

This collaboration is afforded by ENABLE’s slow work of trust building with key allies in AITech, and savvy maneuvering of the precarious political economy of disability in China. Initially, many clients were skeptical that blind people could label texts. ENABLE started with two blind annotators testing the tasks, optimizing the workflow, and modifying accessibility. Piecing together resources from governmental and philanthropic actors, such as subsidies for office space and donations for trainings, while drawing on years of experience in running information technology work programs, ENABLE made the data annotation program viable and competitive. After a year of comparing annotators from ENABLE and other non-disabled contractors, it became clear to AITech that ENABLE workers provided higher quality services.

To ENABLE, this is a great advocacy proof for the productivity of disabled persons in China, whose citizenship is often tied to their contribution to economic production (Dauncey, 2020). The tech sector can also be mobilized as an ally to the increasingly under-resourced NGOs in China (Huang, 2022). ENABLE therefore saw the “feel-good” corporate promotions as an opportunity to leverage the charisma and resource of AI to showcase disability value. However, despite supportive internal advocates, some people in AITech were initially unconvinced of the collaboration’s promotional value. They worried that if the world knows that a bunch of disabled folks are building their systems, users will question the quality of their products. AITech’s users are China’s urban, middle-class, young professionals, who call upon smart home technologies to organize their own overworked, fast-paced lives. The presence of disabled workers, it seems, may threaten the image of the frictionless, efficient, and competent virtual service worker who always obediently stays out of sight (Atanasoski and Vora, 2015).

After years of working closely together, AITech has moved from treating ENABLE like a “small experiment” to recognizing their value across the company. At a dinner I joined, workers spoke of a recent visit to AITech campus as a milestone. The pride and joy of finally earning respect and building genuine trust with powerful corporate actors cannot be read merely cynically. AITech’s senior leadership is recommending ENABLE to more teams and even other tech companies, which, to the workers, marks an appreciation of their value, and a success of “counter-eugenics activism” (Garland-Thomson, 2012). While assisting the technology through their labor and expertise, ENABLE and many disabled workers also sought to co-opt resources for community gain through hazardous engagement with techno-capitalism in uneven power terrains.

In sum, thanks to disability expertise, annotating at ENABLE turned out to be a better option for many disabled workers than being a massage therapist, doing digital piecework at home, or being excluded by non-disabled colleagues in mainstream workplaces. ENABLE’s access work (Hickman, 2019) improved many annotators’ work experience and work performance. Being in a vocal community about disabled people’s capabilities also adds a sense of purpose to their labor. Of course, ENABLE’s workers only constitute a small elite of disabled Chinese, and the DPO had to turn down many eager job candidates and select particular kinds of workers to keep the business viable. Nevertheless, these exceptional individuals have the potential to redistribute their gained advantages and renew visions for community betterment (Mauksch, 2021). I strive here to spotlight the pragmatic efforts and creative workarounds that Chinese DPOs make to further their advocacy agenda under an increasingly hostile political environment (Huang, 2019). Prioritizing workers’ experience and enacting collective bargaining, ENABLE’s practices can also offer a model for cooperative ownership and worker-governed “platforms” that digital labor scholars are advocating for (Vallas and Schor, 2020; Posada, 2021).

 

++++++++++

Conclusion

Contrary to mainstream corporate mythologies of “tech for good,” this research finds disability “good for tech” in profound ways. In this study, disabled data annotators offer a stable and high-quality human resource to the making of smart home AI systems, so that ambiguous human intentions and unruly human speech could be rendered legible to machines in consistent ways. Disabled people’s labor is competitive in this context, not because they are “naturally” more stable than non-disabled workers, but because they are pushed out of the mainstream job market by structural ableism, and pulled into ENABLE’s data annotation workforce by its accommodating labor practices. Reconfiguring space, time, and political economy, ENABLE and the workers’ disability expertise actively transformed the conditions of “ghost work,” and made their work more valuable than other suppliers.

This exceptional case is made possible because it is led by disabled people themselves, as opposed to typical commercial or governmental programs. I highlight a kind of disability expertise that I call “resource hacking,” namely, disability-informed, pragmatic skills that optimize resources in precarious political economies for community betterment. The DPO and disabled workers strategically leveraged the charisma of AI to showcase the productivity of disabled people, a form of counter-eugenics activism (Garland-Thomson, 2012) not without risks of co-optation. In turn, the AI company benefits from the skilled labor, naturalized immobility, and collective expertise of the disabled workers.

Rather than a simple story of extractive capitalism, this article strives to illuminate the potential of disability expertise in renegotiating the terms of techno-capitalism. I draw attention to disabled people’s underexamined role as technicians in information systems, in which disability is often conceived as a problem (Shew, 2020a), a pretext (Mills, 2010), or an afterthought (Nakamura, 2019). An intervention of this article is not only to lay bare the use and abuse of disability as a resource in contemporary AI development, but also to elevate crip technoscience by teasing out the disability expertise actually entailed in the production of AI. I foreground forms of disability value-making that build on genuine expertise, and are thoughtfully constructed with an activist agenda.

As an organization with deep knowledge and lived experience of disability, ENABLE negotiates on behalf of the workers, accommodates heterogenous temporal preferences at work, and builds workspace like a community space, despite scarce resources. They show that labor relations in information technology can be otherwise. Reshaping systems to meet the human, rather than vice versa, workplace disability expertise operationalizes more equitable labor practices in AI. End of article

 

About the author

Di Wu is a Ph.D. candidate in the History, Anthropology, and Science, Technology, and Society (HASTS) program at MIT. Her research and practice focus on the relationship between disability, labor, and technology in contemporary China.
E-mail: wdi [at] mit [dot] edu

 

Acknowledgements

I am deeply grateful to my interlocutors who generously shared their stories with me. Many drafts of this article also benefited from the significant input of Gaoshan Fu, Stefan Helmreich, Andrew Hundt, Graham M. Jones, Crystal Lee, Timothy Y. Loh, Darius Longarino, Rui Wang, Ruoqi Zhu, as well as the anonymous reviewers and the editors of this special issue of First Monday. This study is in part supported by the American Ethnological Society, University of Cambridge Mellon Sawyer Seminar on “Histories of AI: A genealogy of power,” Wenner-Gren Foundation Dissertation Fieldwork Grant (Gr. 10357), and National Science Foundation Doctoral Dissertation Research Improvement Grant (BCS-2213722).

All companies, organizations, and individuals involved in this study are given pseudonyms, and personal identities are rearranged to protect anonymity to the extent possible. Although public information exists about the program examined, and the identity of some parties may be recognizable to those familiar with the issue, I strive to minimize representation that could bring economic or political risks to my interlocutors. Sources that contain identifiable information are not cited to protect anonymity. All Chinese to English translations are mine.

 

Notes

1. Hartblay, 2020, p. S34.

2. Hamraie, 2017, p. xiii.

3. Kafer, 2013, p. 27.

4. Katzman, et al., 2020, p. 521.

 

References

Moritz Altenried, 2020. “The platform as factory: Crowdwork and the hidden labour behind artificial intelligence,” Capital & Class, volume 44, number 2, pp. 145–158.
doi: https://doi.org/10.1177/0309816819899410, accessed 12 December 2022.

Neda Atanasoski and Kalindi Vora, 2015. “Surrogate humanity: Posthuman networks and the (racialized) obsolescence of labor,” Catalyst: Feminism, Theory, Technoscience, volume 1, number 1, pp. 1–40.
doi: https://doi.org/10.28968/cftt.v1i1.28809, accessed 12 December 2022.

Cynthia L. Bennett, Daniela K. Rosner, and Alex S. Taylor, 2020. “The care work of access,” CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15.
doi: https://doi.org/10.1145/3313831.3376568, accessed 12 December 2022.

Tom Boellstorff, 2019. “The opportunity to contribute: Disability and the digital entrepreneur,” Information, Communication & Society, volume 22, number 4, pp. 474–490.
doi: https://doi.org/10.1080/1369118X.2018.1472796, accessed 12 December 2022.

Geoffrey C. Bowker and Susan Leigh Star, 1999. Sorting things out: Classification and its consequences. Cambridge, Mass.: MIT Press.
doi: https://doi.org/10.7551/mitpress/6352.001.0001, accessed 12 December 2022.

Kai Cheng, 2021. “建立巩固拓展残疾人脱贫攻坚成果长效机制 [Establishing long-term mechanisms to solidify and expand the outcomes of the tough battles of poverty elimination for disabled people],” qstheory.cn, at http://www.qstheory.cn/dukan/hqwg/2021-09/28/c_1127912883.htm, accessed 25 April 2022.

China Disabled Persons’ Federation, 2021. “2020年残疾人事业发展统计公报 [2020 official statistics for the development of disabled people’s affairs],” at http://www.caoss.org.cn/sbnr.asp?id=2238, accessed 25 April 2022.

Chinanews.com, 2019. “中国残疾人就业率稳步提升 [Employment rate of Chinese disabled people is steadily increasing],” at http://www.chinanews.com.cn/sh/2019/08-02/8915152.shtml, accessed 25 April 2022.

Julie Y. Chu, 2010. Cosmologies of credit: Transnational mobility and the politics of destination in China. Durham, N.C.: Duke University Press.
doi: https://doi.org/10.1515/9780822393160, accessed 12 December 2022.

Fengming Cui, Cai Cong, Xu Qiaoxian, and Xiaoming Chang, 2019. “Equal participation of persons with disabilities in the development of disability policy on accessibility in China,” International Journal of Developmental Disabilities, volume 65, number 5, pp. 319–326.
doi: https://doi.org/10.1080/20473869.2019.1664842, accessed 12 December 2022.

Sarah Dauncey, 2020. Disability in contemporary China: Citizenship, identity and culture. Cambridge: Cambridge University Press.
doi: https://doi.org/10.1017/9781316339879, accessed 12 December 2022.

Kerry Dobransky and Eszter Hargittai, 2006. “The disability divide in Internet access and use,” Information, Communication & Society, volume 9, number 3, pp. 313–334.
doi: https://doi.org/10.1080/13691180600751298, accessed 12 December 2022.

Arseli Dokumaci, 2020. “People as affordances: Building disability worlds through care intimacy,” Current Anthropology, volume 61, number S21, pp. S97–S108.
doi: https://doi.org/10.1086/705783, accessed 12 December 2022.

Hamid R. Ekbia and Bonnie A. Nardi, 2017. Heteromation, and other stories of computing and capitalism. Cambridge, Mass.: MIT Press.

Elizabeth Ellcessor, 2016. Restricted access: Media, disability, and the politics of participation. New York: New York University Press.
doi: https://doi.org/10.18574/nyu/9781479813803.001.0001, accessed 12 December 2022.

Carla Freeman, 2000. High tech and high heels in the global economy: Women, work, and pink-collar identities in the Caribbean. Durham, N.C.: Duke University Press.
doi: https://doi.org/10.1215/9780822380290, accessed 12 December 2022.

Michele Friedner, 2015a. “Deaf bodies and corporate bodies: New regimes of value in Bangalores business process outsourcing sector,” Journal of the Royal Anthropological Institute, volume 21, number 2, pp. 313–329.
doi: https://doi.org/10.1111/1467-9655.12208, accessed 12 December 2022.

Michele Friedner, 2015b. Valuing deaf worlds in urban India New Brunswick, N.J.: Rutgers University Press.

Michele Friedner, 2014. “Deaf capital: An exploration of the relationship between stigma and value in deaf multilevel marketing participation in urban India,” Medical Anthropology Quarterly, volume 28, number 4, pp. 502–518.
doi: https://doi.org/10.1111/maq.12115, accessed 12 December 2022.

Rosemarie Garland-Thomson, 2012. “The case for conserving disability,” Journal of Bioethical Inquiry, volume 9, number 3, pp. 339–355.
doi: https://doi.org/10.1007/s11673-012-9380-0, accessed 12 December 2022.

Faye Ginsburg and Rayna Rapp, 2013. “Disability worlds,” Annual Review of Anthropology, volume 42, pp. 53–68.
doi: https://doi.org/10.1146/annurev-anthro-092412-155502, accessed 12 December 2022.

Gerard Goggin and Katie Ellis, 2020. “Disability, communication, and life itself in the COVID-19 pandemic,” Health Sociology Review, volume 29, number 2, pp. 168–176.
doi: https://doi.org/10.1080/14461242.2020.1784020, accessed 12 December 2022.

Gerard Goggin, Haiqing Yu, Karen R. Fisher, and Bingqin Li, 2019. “Disability, technology innovation and social development in China and Australia,” Journal of Asian Public Policy, volume 12, number 1, pp. 34–50.
doi: https://doi.org/10.1080/17516234.2018.1492067, accessed 12 December 2022.

Mary L. Gray and Siddharth Suri, 2019. Ghost work: how to stop Silicon Valley from building a new global underclass, Boston, Mass.: Houghton Mifflin Harcourt.

Aimi Hamraie, 2017. Building access: Universal design and the politics of disability. Minneapolis: University of Minnesota Press.

Aimi Hamraie and Kelly Fritsch, 2019. “Crip technoscience manifesto,” Catalyst: Feminism, Theory, Technoscience, volume 5, number 1, pp. 1–33.
doi: https://doi.org/10.28968/cftt.v5i1.29607, accessed 12 December 2022.

Cassandra Hartblay, 2020. “Disability expertise: Claiming disability anthropology,” Current Anthropology, volume 61, number S21, pp. S26–S36.
doi: https://doi.org/10.1086/705781, accessed 12 December 2022.

Lucia Hasty, n.d. “Teaching tactile graphics,” Perkins School for the Blind eLEARNING, at https://www.perkinselearning.org/videos/webcast/teaching-tactile-graphics, accessed 28 March 2021.

Stefan Helmreich, 2007. “An anthropologist underwater: Immersive soundscapes, submarine cyborgs, and transductive ethnography,” American Ethnologist, volume 34, number 4, pp. 621–641.
doi: https://doi.org/10.1525/ae.2007.34.4.621, accessed 12 December 2022.

Sara Hendren, 2020. What can a body do? How we meet the built world. New York: Riverhead Books.

Louise Hickman, 2019. “Transcription work and the practices of crip technoscience,” Catalyst: Feminism, Theory, Technoscience, volume 5, number 1, pp. 1–10.
doi: https://doi.org/10.28968/cftt.v5i1.32081, accessed 12 December 2022.

Luanjiao Hu, 2022. “Opportunities and challenges: An examination of educational landscape for people with visual impairment in China,” Current Issues in Comparative Education, volume 24, number 1.
doi: https://doi.org/10.52214/cice.v21i1.8844, accessed 12 December 2022.

Luanjiao Hu and Jing Lin, 2017. “Access to higher education for people with disabilities: A Chinese perspective,” In: Henry C. Alphin Jr., Roy Y. Chan, and Jennie Lavine (editors). The future of accessibility in international higher education. Hershey, Pa.: IGI Global, pp. 70–87.
doi: https://doi.org/10.4018/978-1-5225-2560-8.ch005, accessed 12 December 2022.

Jin Huang, Baorong Guo, and John C. Bricout, 2009. “From concentration to dispersion: The shift in policy approach to disability employment in China,” Journal of Disability Policy Studies, volume 20, number 1, pp. 46–54.
doi: https://doi.org/10.1177/1044207308325008, accessed 12 December 2022.

Shixin Huang, 2022. “NGO as sympathy vendor or public advocate? A case study of NGOs’ participation in Internet fundraising campaigns in China,” VOLUNTAS, volume 33, pp. 1,064–1,076.
doi: https://doi.org/10.1007/s11266-022-00463-w, accessed 18 March 2022.

Shixin Huang, 2019. “Ten years of the CRPD’s adoption in China: Challenges and opportunities,” Disability & Society, volume 34, number 6, pp. 1,004–1,009.
doi: https://doi.org/10.1080/09687599.2019.1604384, accessed 12 December 2022.

Sarah S. Jain, 1999. “The prosthetic imagination: Enabling and disabling the prosthesis trope,” Science, Technology, & Human Values, volume 24, number 1, pp. 31–54.
doi: https://doi.org/10.1177/016224399902400103, accessed 12 December 2022.

JD.com, 2018. “央视点赞!人工智能为残疾人就业带来新岗位 [Liked by CCTV! AI brings new jobs for disabled people’s employment],” at https://www.sohu.com/a/226893666_99967243, accessed 2 December 2020.

Alison Kafer, 2021. “After crip, crip afters,” South Atlantic Quarterly, volume 120, number 2, pp. 415–434.
doi: https://doi.org/10.1215/00382876-8916158, accessed 12 December 2022.

Alison Kafer, 2013. Feminist, queer, crip. Bloomington: Indiana University Press.

Erika R. Katzman, Elizabeth Anne Kinsella, and Jessica Polzer, 2020. “‘Everything is down to the minute’: Clock time, crip time and the relational work of self-managing attendant services,” Disability & Society, volume 35, number 4, pp. 517–541.
doi: https://doi.org/10.1080/09687599.2019.1649126, accessed 12 December 2022.

Os Keyes, 2020. “Automating autism: Disability, discourse, and artificial intelligence,” Journal of Sociotechnical Critique, volume 1, number 1, pp. 1–31, at https://digitalcommons.odu.edu/sociotechnicalcritique/vol1/iss1/8, accessed 3 June 2021.

Gary Klein, Jennifer K. Phillips, Erica L. Rall, and Deborah A. Peluso, 2007. “A data-frame theory of sensemaking,” In: Robert R. Hoffman (editor). Expertise out of context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. Mahwah, N.J.: Lawrence Erlbaum Associates, pp. 113–155.

Matthew Kohrman, 2005. Bodies of difference: Experiences of disability and institutional advocacy in the making of modern China. Berkeley: University of California Press.

Tamar Lavee, Lili Kotlerman, Matan Orbach, Yonatan Bilu, Michal Jacovi, Ranit Aharonov, and Noam Slonim, 2019. “Crowd-sourcing annotation of complex NLU tasks: A case study of argumentative content annotation,” Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP, pp. 29–38.
doi: https://doi.org/10.18653/v1/D19-5905, accessed 12 December 2022.

Xiaochang Li and Mara Mills, 2019. “Vocal features: From voice identification to speech recognition by machine,” Technology and Culture, volume 60, number S2, pp. S129–S160.
doi: https://doi.org/10.1353/tech.2019.0066, accessed 12 December 2022.

Xue Hui Li, Su Qiong Xu and Luan Jiao Hu, 2022. “Protection or restriction: An analysis of the blind massage policy for employment rights of persons with visual impairments in China,” Journal of Visual Impairment & Blindness, volume 116, number 1, pp. 85–95.
doi: https://doi.org/10.1177/0145482X211073611, accessed 12 December 2022.

Xuehui Li and Zhijun Fu, 2015. “从隔离走向融合? — 我国视力障碍者高等教育的历史特点及政策变迁 [From isolation to integration — The history, features and policy change of people with visual impairment higher education],” Journal of Social Work, number 6, pp. 35–46.

Juan Liao, 2021. “The quota system for employment of people with disabilities in China: Policy, practice, barriers, and ways forward,” Disability & Society, volume 36, number 2, pp. 326–331.
doi: https://doi.org/10.1080/09687599.2020.1833311, accessed 12 December 2022.

Zhongxuan Lin and Liu Yang, 2020. “A digital promised land? Digital landscape as a heterotopia for disabled people in China,” Information, Communication & Society, volume 23, number 8, pp. 1,220–1,234.
doi: https://doi.org/10.1080/1369118X.2020.1776366, accessed 12 December 2022.

Zhongxuan Lin, Zhi’an Zhang, and Liu Yang, 2019. “Self as enterprise: Digital disability practices of entrepreneurship and employment in the wave of ‘Internet + disability’ in China,” Information, Communication & Society, volume 22, number 4, pp. 554–569.
doi: https://doi.org/10.1080/1369118X.2018.1518470, accessed 12 December 2022.

Alan Lundgard, Crystal Lee, and Arvind Satyanarayan, 2019. “Sociotechnical considerations for accessible visualization design,” 2019 IEEE Visualization Conference (VIS), pp. 16–20.
doi: https://doi.org/10.1109/VISUAL.2019.8933762, accessed 12 December 2022.

Caitrin Lynch, 2013. “Membership and mattering: Agency and work in a New England factory,” In: Caitrin Lynch and Jason Danely (editors). Transitions and transformations: Cultural perspectives on aging and the life course. New York: Berghahn Books, pp. 188–205.
doi: https://doi.org/10.1515/9780857457790-015, accessed 12 December 2022.

Stefanie Mauksch, 2021. “Being blind, being exceptional: Work integration, social entrepreneurship and the reimagination of blind potential in Nepal,” Disability & Society (9 July).
doi: https://doi.org/10.1080/09687599.2021.1933387, accessed 12 December 2022.

Milagros Miceli, Martin Schuessler, and Tianling Yang, 2020. “Between subjectivity and imposition: Power dynamics in data annotation for computer vision,” Proceedings of the ACM on Human-Computer Interaction, volume 4, number CSCW2, article number 115, pp. 1–25.
doi: https://doi.org/10.1145/3415186, accessed 12 December 2022.

Mara Mills, 2010. “Deaf jam: From inscription to reproduction to information,” Social Text, volume 28, number 1, pp. 35–58.
doi: https://doi.org/10.1215/01642472-2009-059, accessed 12 December 2022.

Mia Mingus, 2010. “Interdependency (excerpts from several talks),” Leaving Evidence (22 January), at https://leavingevidence.wordpress.com/2010/01/22/interdependency-exerpts-from-several-talks/, accessed 28 March 2021.

David Mitchell and Sharon Snyder, 2010. “Disability as multitude: Re-working non-productive labor power,” Journal of Literary & Cultural Disability Studies, volume 4, number 2, pp. 179–194.
doi: https://doi.org/10.3828/jlcds.2010.14, accessed 12 December 2022.

David Mitchell and Sharon Snyder, 2001. Narrative prosthesis: Disability and the dependencies of discourse. Ann Arbor: University of Michigan Press.
doi: https://doi.org/10.3998/mpub.11523, accessed 12 December 2022.

Karen Nakamura, 2019. “My algorithms have determined youre not human: AI-ML, reverse Turing-tests, and the disability experience,” ASSETS ’19: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, pp. 1–2.
doi: https://doi.org/10.1145/3308561.3353812, accessed 20 October 2021.

Lisa Nakamura, 2014. “Indigenous circuits: Navajo women and the racialization of early electronic manufacture,” American Quarterly, volume 66, number 4, pp. 919–941.
doi: https://doi.org/10.1353/aq.2014.0070, accessed 12 December 2022.

Netease.com, 2018. “人工智能与‘她’‘京东微工’助残疾人逆光飞翔 [AI and ‘her’: Jingdong Microwork helps disabled people fly against the light],” at https://art.163.com/18/0530/14/DJ2H5KV4009998F3.html, accessed 2 December 2020.

Julian Posada, 2021. “Family units: Can the families whose labor powers AI seize the means of production?” Logic, number 15 (Beacons), at https://logicmag.io/beacons/family-units/, accessed 18 March 2022.

Yuanyuan Qu, 2022. “Is the Internet the game changer? Disabled people and digital work in China,” Disability & Society, volume 37, number 5, pp. 725–745.
doi: https://doi.org/10.1080/09687599.2020.1833314, accessed 12 December 2022.

Yuanyuan Qu, 2020. “Understanding the body and disability in Chinese contexts,” Disability & Society, volume 35, number 5, pp. 738–759.
doi: https://doi.org/10.1080/09687599.2019.1649123, accessed 12 December 2022.

Kathleen Richardson, 2015. An anthropology of robots and AI: Annihilation anxiety and machines. New York: Routledge.
doi: https://doi.org/10.4324/9781315736426, accessed 12 December 2022.

Holly Robbins, Taylor Stone, John Bolte, and Jeroen van den Hoven, 2021. “Legibility as a design principle: Surfacing values in sensing technologies,” Science, Technology, & Human Values, volume 46, number 5, pp. 1,104–1,135.
doi: https://doi.org/10.1177/0162243920975488, accessed 12 December 2022.

Sarah T. Roberts, 2019. Behind the screen: Content moderation in the shadows of social media. New Haven, Conn.: Yale University Press.

Ellen Samuels and Elizabeth Freeman, 2021. “Introduction: Crip temporalities,” South Atlantic Quarterly, volume 120, number 2, pp. 245–254.
doi: https://doi.org/10.1215/00382876-8915937, accessed 12 December 2022.

Beth M. Semel, 2022. “Listening like a computer: Attentional tensions and mechanized care in psychiatric digital phenotyping,” Science, Technology, & Human Values, volume 47, number 2, pp. 266–290.
doi: https://doi.org/10.1177/01622439211026371, accessed 12 December 2022.

Ashley Shew, 2020a. “Ableism, Technoableism, and future AI,” IEEE Technology and Society Magazine, volume 39, number 1, pp. 40–85.
doi: https://doi.org/10.1109/MTS.2020.2967492, accessed 12 December 2022.

Ashley Shew, 2020b. “From a figment of your imagination: Disabled marginal cases and underthought experiments,” Human Affairs, volume 30, number 4, pp. 608–616.
doi: https://doi.org/10.1515/humaff-2020-0054, accessed 12 December 2022.

Jonathan Sterne, 2003. The audible past: Cultural origins of sound reproduction. Durham, N.C.: Duke University Press.
doi: https://doi.org/10.1215/9780822384250, accessed 12 December 2022.

Jonathan Sterne and Mara Mills, 2020. “Second rate,” Triple Canopy, at https://www.canopycanopycanopy.com/contents/second-rate, accessed 16 March 2021.

Lucy Suchman, 1993. “Do categories have politics? The language/action perspective reconsidered,” In: Giorgio Michelis, Carla Simone, and Kjeld Schmidt (editors). Proceedings of the Third European Conference on Computer-Supported Cooperative Work 13–17 September 1993, Milan, Italy ECSCW ’93. Dordrecht: Springer Netherlands, pp. 1–14.
doi: https://doi.org/10.1007/978-94-011-2094-4_1, accessed 12 December 2022.

Shan Tie, Rong Guo, and Xi Chen, 2011. “回忆与思考 — 谈中国盲人按摩的发展历程 [Reminiscence and reflection on the development of Chinese blind massage],” Disability in China, number 6, pp. 46–48.

Paola Tubaro, Antonio A Casilli, and Marion Coville, 2020. “The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence,” Big Data & Society, volume 7, number 1.
doi: https://doi.org/10.1177/2053951720919776, accessed 12 December 2022.

Steven Vallas and Juliet B. Schor, 2020. “What do platforms do? Understanding the gig economy,” Annual Review of Sociology, volume 46, pp. 273–294.
doi: https://doi.org/10.1146/annurev-soc-121919-054857, accessed 12 December 2022.

Di Wu, 2021. “Cripping the history of computing,” IEEE Annals of the History of Computing, volume 43, number 3, pp. 68–72.
doi: https://doi.org/10.1109/MAHC.2021.3101061, accessed 12 December 2022.

Anon Ymous, Katta Spiel, Os Keyes, Rua M. Williams, Judith Good, Eva Hornecker, and Cynthia L. Bennett, 2020. “‘I am just terrified of my future’ — Epistemic violence in disability related technology research,” CHI EA ’20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16.
doi: https://doi.org/10.1145/3334480.3381828, accessed 12 December 2022.

Haiqing Yu, Gerard Goggin, Karen Fisher, and Bingqin Li, 2019. “Introduction: disability participation in the digital economy,” Information, Communication & Society, volume 22, number 4, pp. 467–473.
doi: https://doi.org/10.1080/1369118X.2018.1550525, accessed 12 December 2022.

Sean Zdenek, 2015. Reading sounds: Closed-captioned media and popular culture. Chicago: University of Chicago Press.

Kathryn Zyskowski, Meredith Ringel Morris, Jeffrey P. Bigham, Mary L. Gray, and Shaun K. Kane, 2015. “Accessible crowdwork? Understanding the value in and challenge of microtask employment for people with disabilities,” CSCW ’15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp. 1,682–1,693.
doi: https://doi.org/10.1145/2675133.2675158, accessed 10 March 2021.

 


Editorial history

Received 16 November 2022; accepted 12 December 2022.


Creative Commons License
This paper is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Good for tech: Disability expertise and labor in China’s artificial intelligence sector
by Di Wu.
First Monday, Volume 28, Number 1 - 2 January 2023
https://firstmonday.org/ojs/index.php/fm/article/download/12887/10754
doi: https://dx.doi.org/10.5210/fm.v28i1.12887