PaperlayerView exact revision on arXiv

Source-linked AI summary

Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibility

David Gray Widder, Dawn Nafus

arXiv:2209.09780v3cs.CYcs.HCcs.SE

TL;DR

Responsible AI guidelines ask engineers to consider potential harms, while AI systems are assembled from modules handled by many people. Through interviews with 27 AI engineers, the paper examines how this distributed work shapes responsibility. Participants often recognized possible harms but saw addressing them as someone else’s responsibility.

  • Problem

    The paper asks how responsibility and agency are socially organized for AI practitioners working across distributed arrangements where engineers may be far from deployment.

    Source: Introduction and background · p. 1

  • Method

    The authors use semi-structured interviews with 27 AI practitioners across industry, open source, and academia to examine how they scope responsibility for possible AI harms.

    Source: Introduction and background · p. 3

  • Results

    Participants acknowledged possible harms but often saw addressing them as beyond their agency, capability, or responsibility, producing a dislocated sense of accountability.

    Source: Introduction and background · p. 1

  • Takeaways & Limitations

    The authors conclude that AI governance depends on practitioners recognizing their locations in broader production systems and engaging in the relations those systems involve.

    Source: Conclusion · p. 10

    The authors caution that this approach could leave ethics a second-order, lagging concern when market value cannot be found.

    Source: Strengthening the interfaces · p. 9

Abstract

from arXiv · show

Responsible artificial intelligence guidelines ask engineers to consider how their systems might harm. However, contemporary artificial intelligence systems are built by composing many preexisting software modules that pass through many hands before becoming a finished product or service. How does this shape responsible artificial intelligence practice? In interviews with 27 artificial intelligence engineers across industry, open source, and academia, our participants often did not see the questions posed in responsible artificial intelligence guidelines to be within their agency, capability, or responsibility to address. We use Suchman's "located accountability" to show how responsible artificial intelligence labor is currently organized and to explore how it could be done differently. We identify cross-cutting social logics, like modularizability, scale, reputation, and customer orientation, that organize which responsible artificial intelligence actions do take place and which are relegated to low status staff or believed to be the work of the next or previous person in the imagined "supply chain." We argue that current responsible artificial intelligence interventions, like ethics checklists and guidelines that assume panoptical knowledge and control over systems, could be improved by taking a located accountability approach, recognizing where relations and obligations might intertwine inside and outside of this supply chain.

Introduction and background

The paper examines how modular software production shapes AI developers’ sense of responsibility for harms, finding that harms may be acknowledged yet treated as someone else’s job. It argues for located accountability: recognizing partial knowledge and working relations across boundaries rather than assuming developers can anticipate every outcome.

Introduction and background

  • Developers acknowledged possible AI harms but often viewed addressing them as beyond their agency, capability, or responsibility, leaving accountability dislocated across the imagined supply chain.The authors argue that modularity and its divisions of labor reinforce this pattern, while harm recognition often occurs through social locations that cross or sit outside the chain.Source: Introduction and background · p. 1
  • The authors argue that checklists, model cards, and data sheets that map technology to end use may conflict with software-production realities and assumptions of total knowledge and control.The paper concludes by proposing three paths forward depending on how deeply one is prepared to question modularity.Source: Introduction and background · pp. 2–3
  • Modularity divides software into self-contained parts, facilitating information hiding and specialized labor while making relationships among developers, users, and affected citizens easier to bracket off.The resulting situational ignorance is not total: participants were aware of harm, usually when outside their software-engineering role.Source: Introduction and background · p. 2
  • The paper examines how modularity organizes accountability alongside cross-cutting forces such as business relations, reputation, and user experience, which developers may treat as outside the supply chain.It frames these secondary relations as potential sites where working relations can be recognized and responsible action enabled.Source: Introduction and background · p. 2
  • The authors propose located accountability: developers should bring partial, situated knowledge to working relations across module boundaries rather than be expected to anticipate every outcome through exhaustive checklists.They argue boundary-crossing can reduce ethical debt, and identify social locations that could enable it.Source: Introduction and background · p. 3
  • The study draws on semi-structured interviews with 27 AI practitioners recruited across industry, open source, and academia, representing varied positions along the supply chain.Participants were not connected through one traceable supply chain; the interviews and analysis examined how they positioned work from general-purpose to specific-use systems.Source: Introduction and background · p. 3

Views from up and down the AI supply chain

Developers’ accounts show that supply-chain position shapes what they see as controllable and responsible: accountability is often displaced elsewhere, even as downstream developers recognize concerns about misuse, suppliers, and customers.

Views from up and down the AI supply chain

  • High in the chain, developers treated general-purpose tools as technologically neutral, overlooking imagined users and their own role in enabling harmful machine-learning uses.The researcher described abuse as a general possibility and compared his contribution to pipe-making equipment later used to make a gun barrel.Source: Views from up and down the AI supply… · p. 4
  • A middle-chain benchmark developer disclaimed responsibility for an open-source framework, prioritized performance numbers over translation meaning, and deferred bias concerns to the deploying client.He expected the next person in the chain to address bias, while describing data and components as already available or prepared.Source: Views from up and down the AI supply… · p. 4
  • A lower-chain tech lead worried about code flaws, security, and reputational damage, yet welcomed program managers buffering engineers from customer interactions and social uses.The separation could insulate engineers from knowing that their software might train military drone pilots.Source: Views from up and down the AI supply… · pp. 4–5
  • A downstream developer saw design limits on accidental misuse but worried about dependence on Facebook’s Oculus platform and recognized the need to qualify customers.The participant feared inheriting a supplier’s ethical debt; customer qualification had not come up higher in the supply chain.Source: Views from up and down the AI supply… · p. 5
  • Modularity makes responsibility seem absent “here” wherever developers are, sustaining partial ignorance of suppliers and customers and displacing context elsewhere.The authors describe this as a view from within a digital cargo container, shaped both by the difficulty of knowing across many hands and by little desire to know.Source: Views from up and down the AI supply… · p. 5

Crosscurrents within and against the supply chain

The section examines how the supply chain is reproduced and how people step outside it to prevent harm, through both institutionally sanctioned and unsanctioned actions.

Crosscurrents within and against the supply chain

Reproducing the supply chain

Divided labor and the pursuit of scale make relationships and responsibilities across the AI supply chain harder to sustain. Participants described ethics work as unassigned, low-status, or beyond their authority, while scale could justify losing contact with customers and their uses of AI.

Reproducing the supply chain

  • Participants described ethics work as unassigned or outside their role, leaving relationships and responsibility to be divided or handed off.One engineer said responsible AI was not part of the work assigned from the top; a UX researcher questioned why engineers filled out ethics assessments instead of people who talk to customers.Source: Reproducing the supply chain · p. 5
  • When ethics became a task, participants described it as low-status administrative work passed to contractors or junior employees.One engineer found a privacy questionnaire asked about storage security that depended on IT support and configuration.Source: Reproducing the supply chain · p. 5
  • Participants said research incentives privileged results over societal impact, while ethics assessments blurred the boundary of engineers’ authority.A university researcher said conference audiences usually wanted results, while a body-scanning developer found it confusing to answer questions about the broader product and business.Source: Reproducing the supply chain · p. 6
  • Participants saw scaling as making customers and their uses of AI harder to know, weakening the personal relationships that connected supply-chain modules.One participant said tracking clients’ behavior would be difficult after the company went public and had 10,000 clients.Source: Reproducing the supply chain · p. 6
  • Some participants used growth and scale to justify deprioritizing ethical AI, fearing it could lose customers, create bottlenecks, or limit technological progress.One participant said ethical AI was not a major concern while the company focused on growing and scaling users.Source: Reproducing the supply chain · p. 6
  • The authors interpret scale as a perceived moral imperative that can sever care relationships and make harm easier not to see.They connect scale thinking to modularity and capitalist impulses (Hanna and Park, 2020), and describe technologies being cut loose from production sites for mass export (Suchman, 2002: 95).Source: Reproducing the supply chain · p. 6

Acting outside the supply chain

Although developers often described responsible action as outside their supply-chain role, customer, reputational, workplace, and personal ties connected them to downstream impacts. These connections coexisted with modular limits that favored some harms for attention while leaving other ethical work informal or invisible.

Acting outside the supply chain

  • Developers’ customer and supplier connections could surface downstream impacts, but customer-centered design often privileged paying customers over other affected parties.Anticipated users linked commercial incentives with possible societal impacts, while customer contact was also seen as a way to learn about downstream problems.Source: Acting outside the supply chain · pp. 6–7
  • Participants saw responsible AI as a market differentiator, while reputational concerns could prompt companies to sever downstream ties and reflected economic consequences, social obligations, or both.Developers also feared that harmful uses of their work could damage their own reputations; participants questioned whether some corporate ethics commitments were sincere when profits and ethics conflicted.Source: Acting outside the supply chain · p. 7
  • Developers’ personal relationships made downstream harms harder to disavow, and one participant wanted to know where and for what purpose his software was used.Questions from friends and family about AI harms confronted one developer with the uses of general-purpose software; another described not knowing whether a framework was used by the US Army as itself a kind of harm.Source: Acting outside the supply chain · pp. 7–8
  • Developers quietly resisted a garment-worker tracking project by deprioritizing it until the client withdrew, an offstage ethical action absent from official responsible AI talk.The participant worried that motion tracking would amount to algorithmic management of workers.Source: Acting outside the supply chain · p. 8
  • Attention favored public harms that fit modular work, such as measurable bias, while broader questions about weapon use went unaddressed; accuracy checks and paperwork coexisted with invisible ethical labor.The section describes a split between prominent public dramas, contractor paperwork, limited accuracy checks, and offstage actions such as slowing work or caring for colleagues.Source: Acting outside the supply chain · p. 8

Where to go from here?

The authors argue that responsible-AI tools often assume a level of visibility and control that developers lack. They propose starting from situated relations and partial knowledge, while recognizing that adopting located accountability wholesale may conflict with valued modularity.

Where to go from here?

  • Checklists, model cards, datasheets, and inventory-like toolkits assume visibility or control that teams often lack, limiting their likely success.Some tools present facts to an unspecified audience, while teams may have neither visibility nor control over cascades of failure—and may not believe they should.Source: Where to go from here? · p. 8
  • A located-accountability approach would begin with partial, situated knowledge and seek relations between actors, including stronger roles for nondevelopers.Source: Where to go from here? · p. 8
  • Adopting located accountability wholesale is not straightforward because modularity-focused colleagues value cutting relations for specific reasons.The authors identify three possible approaches, depending on how willing colleagues trained in modularity are to question it.Source: Where to go from here? · p. 8

Acting within the modules

The paper proposes acting within modularity by recording developers’ partial knowledge in documentation rather than asking them to make definitive claims. Dividing documentation among roles could bring partial knowledge together, but risks preserving exclusion and leaving unsayable concerns to outsiders.

Acting within the modules

  • Checklists and model cards could record participants’ partial understanding of flaws, limitations, provenance, and use contexts as models and data change hands.This could relieve developers of pressure to definitively claim facts they feel unable to claim.Source: Acting within the modules · p. 8
  • Clearly separating knowledge supplied by developers, users, sales, and legal roles could make model cards boundary objects that combine partial contributions.The approach doubles down on division of labor and leaves the supply-chain metaphor largely intact, without necessarily creating deeper relations.Source: Acting within the modules · p. 8
  • Without a creative way to include impacted groups, this approach re-inscribes their exclusion and limits non-developer contributions to what is publicly sayable.Regulators, journalists, and academics are left to press for conversation and action about concerns considered unsayable within the chain.Source: Acting within the modules · p. 9

Strengthening the interfaces

The section proposes strengthening accountability through value-chain relationships and interfaces that connect actors across supply chains, while warning that market-centered coordination can leave ethics secondary.

Strengthening the interfaces

  • A value-chain approach would strengthen intercompany communication through contractual model-card obligations and customer scrutiny of consented data and data-labeler pay, positioning ethics as customer value.This approach is not equally feasible for every company; some AI entrepreneurs concealed ethics work from venture funders to hide limitations.Source: Strengthening the interfaces · p. 9
  • Accountability also requires stronger interfaces between onstage and offstage actors, integrating multiple supply-chain locations and making reputational costs and multi-actor cascades more visible.Developers might refuse personal reputational costs, while journalists and academics could emphasize multi-actor cascades.Source: Strengthening the interfaces · p. 9
  • Rather than assuming perfect control over each module, value chains can work through probabilities and frictions, including licenses that make harmful software uses harder.The Ethical Source movement uses licenses to add legal friction while acknowledging that control is not total.Source: Strengthening the interfaces · p. 9
  • These arrangements may strengthen norms, but positional misunderstandings can impede alignment, while market-defined value risks leaving ethics a lagging, second-order concern.The risk depends on policy conditions and arises when market value cannot be found for ethics issues.Source: Strengthening the interfaces · p. 9

Rejecting modularity

The section proposes rejecting modularity as both a coding principle and social arrangement: prioritize relationships, build code second, and treat scale as a distant concern.

Rejecting modularity

  • Rejecting modularity means prioritizing relationships over code and treating scale as secondary, with development shaped by context-specific knowledge rather than producer-defined testing grounds.The relationship is the objective, and differences between users matter rather than making one party the testbed for another’s scale.Source: Rejecting modularity · pp. 9–10
  • Indigenous data sovereignty and Gray’s community-health work model this approach through sustained domain connection, accepting friction rather than a rush to scale (Carroll et al., 2020; Gray, 2021; Costanza-Chock, 2020).Gray describes a two-year intensive process for vaccine equity in Black and Latinx communities.Source: Rejecting modularity · p. 10
  • Teams could reject modularity’s broader assumptions or choose upstream components more carefully, revealing that supposedly generic tools serve only those favored by the current supply chain.Developing tools that serve other interests could open engineering paths otherwise foreclosed.Source: Rejecting modularity · p. 10
  • The approach also raises a policy question: how should science and technology policy support resource-poor community groups whose work meets needs that benefit the wider country?The passage contrasts community groups’ limited resources with companies seeking to scale.Source: Rejecting modularity · p. 10

Conclusion

Viewing responsibility as chains of relations reveals where ethical decisions and AI governance interventions can occur. Responsible AI interventions can begin with deliberate choices about how strongly current software-production ideals should shape future development.

  • Responsible AI interventions can begin by deliberately choosing how strongly current software-production ideals should shape future development.Source: Conclusion · p. 10
  • Ethical decision-making can occur at specific upstream or downstream locations, both within and outside modularity’s cultural logic.Source: Conclusion · p. 10
  • Social locations shape what is sayable, prestigious work, paperwork, or public drama, and where AI governance interventions can act.Source: Conclusion · p. 10
  • Governance interventions depend on actors’ willingness and capability to acknowledge their locations in production and engage more fully in their relations.Source: Conclusion · p. 10
  • The role of modularized thinking in software production may not be settled by consensus, and different actors may follow different paths and visions of responsibility.Source: Conclusion · p. 10

Funding

The work was supported by Intel Corporation; accompanying notes explain the authors’ use of “responsible AI” and “ethical AI” and point readers to ethicalsource.dev.

Funding

  • The authors report that Intel Corporation supported the research, authorship, and publication.The disclosure names the Intel Corporation as the source of financial support.Source: Funding · p. 11

Notes

  • The authors use “responsible AI” to express a commitment to feminist technology theories (Haraway, 1991), while using “ethical AI” where context fits.They say “responsible AI” is the more common term and connect ethics to the question of to whom one owes a response.Source: Notes · p. 11
  • A second note directs readers to ethicalsource.dev.Source: Notes · p. 11
Loading 2209.09780v3…