PaperlayerView exact revision on arXiv

Source-linked AI summary

A relationship and not a thing: A relational approach to algorithmic accountability and assessment documentation

Jacob Metcalf, Emanuel Moss, Ranjit Singh, Emnet Tafese, Elizabeth Anne Watkins

arXiv:2203.01455v1cs.CY

TL;DR

The paper asks how publics can access and contest information about algorithmic systems when developers are expected to document their consequences. It develops a relational, triadic accountability framework and argues that assessment regimes should position the public alongside developers and regulators.

  • Problem

    The paper identifies a challenge in accounting for algorithmic systems whose impacts span individuals and collectives, including who can influence how systems are assessed.

    Source: §1 · p. 2, §2 · p. 3, §5 · p. 15

  • Method

    The paper develops a relational accountability approach that places developers or operators, regulators, and publics in a triadic relationship.

    Source: §1 · p. 2

  • Results

    The paper argues that algorithmic accountability should be grounded in a triadic relationship among developers or operators, regulators, and publics.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Algorithmic accountability regulations should prioritize a developer–regulator–public relationship.

    Source: §5 · p. 15

    The mechanisms discussed can draw attention and support public assertion but do not themselves create a legal obligation compelling change.

    Source: §4 · p. 14

Abstract

from arXiv · show

Central to a number of scholarly, regulatory, and public conversations about algorithmic accountability is the question of who should have access to documentation that reveals the inner workings, intended function, and anticipated consequences of algorithmic systems, potentially establishing new routes for impacted publics to contest the operations of these systems. Currently, developers largely have a monopoly on information about how their systems actually work and are incentivized to maintain their own ignorance about aspects of how their systems affect the world. Increasingly, legislators, regulators and advocates have turned to assessment documentation in order to address the gap between the public's experience of algorithmic harms and the obligations of developers to document and justify their design decisions. However, issues of standing and expertise currently prevent publics from cohering around shared interests in preventing and redressing algorithmic harms; as we demonstrate with multiple cases, courts often find computational harms non-cognizable and rarely require developers to address material claims of harm. Constructed with a triadic accountability relationship, algorithmic impact assessment regimes could alter this situation by establishing procedural rights around public access to reporting and documentation. Developing a relational approach to accountability, we argue that robust accountability regimes must establish opportunities for publics to cohere around shared experiences and interests, and to contest the outcomes of algorithmic systems that affect their lives. Furthermore, algorithmic accountability policies currently under consideration in many jurisdictions must provide the public with adequate standing and opportunities to access and contest the documentation provided by the actors and the judgments passed by the forum.

1 INTRODUCTION

As assessment requirements expand, the paper asks who must disclose information about algorithmic systems and who can receive it and demand changes. It argues for relational accountability that gives publics a role alongside developers and regulators.

1 INTRODUCTION

  • Developers may be required to study system consequences, but accountability also depends on who must disclose that information and who can demand changes.The introduction frames access to documentation and the power to make demands as central questions.Source: §1 · p. 2
  • The proposed accountability relationship includes developers or operators, regulators, and publics, rather than only actors and forums.The paper builds on an actor–forum account and says documentation can ground contestation only when the public can make demands.Source: §1 · p. 2
  • The paper develops a relational approach focused on how accountability relations are constituted between actors, forums, and publics.Its stated agenda includes barriers to public participation and how governance regimes can structure relationships to foreground the public interest.Source: §1 · pp. 2–3
  • The paper focuses on courts in the United States, except where comparisons with other jurisdictions are generative.Source: §1 · p. 2

2 ACCOUNTABILITY IS A RELATION, NOT A PROPERTY

The paper treats algorithmic accountability as a relationship, not a property of a system or a document. It argues that accountability must connect developers, regulators, and affected publics, addressing the mismatch between collective harms and individual routes to redress.

2 ACCOUNTABILITY IS A RELATION, NOT A PROPERTY

  • Accountability is constituted through relations among developers, regulators, and publics, not through a system property or static compliance document.The relational approach aims to let publics emerge, cohere, and assert shared interests to obligated regulators and developers.Source: §2 · pp. 3–4
  • A relational account asks how parties are made responsible to one another, rather than treating accountability as a virtue or system feature.The paper contrasts accountability as virtue with accountability as a mechanism involving actors and forums.Source: §2 · p. 3
  • Collective sources of algorithmic harm often meet legal frameworks that recognize individual injuries, leaving collectivities without a way to make demands on developers and regulators.The paper describes harms as originating in collectivities while individual instances are more readily identifiable to plaintiffs with existing rights.Source: §2 · pp. 3–4
  • Individualized remedies can leave group-level harms and their underlying conditions unaddressed.The facial-recognition example contrasts redress for an individual arrest with the absence of a legal pathway to reject biased training data or its historical relations.Source: §2 · p. 4
  • A regulator–developer dyad can sideline impacted communities, while a public third party can counterbalance actors and hold the forum accountable.The paper links dyadic oversight to legal endogeneity and the risk that regulatory priorities shift toward actors’ priorities.Source: §2 · p. 5
  • Documentation of system workings and consequences gives data subjects and publics a route to contest lived impacts through litigation.Courts can provide redress and serve as a backstop for holding forums accountable for the harms they require actors to address.Source: §2 · p. 5
  • Algorithmic impact assessments can create points where the public contests the adequacy and accuracy of developers’ claims.The paper identifies studying and reporting system consequences as part of these governance regimes.Source: §2 · p. 5

3 BUILDING CAPACITY FOR ALGORITHMIC ACCOUNTABILITY

This section examines how to build algorithmic-accountability capacity by shifting power toward publics in a triadic actor–forum–public relationship. It describes how standing, opaque decisions, and gaps in documentation and expertise constrain challenges to algorithmic harms, and considers documentation requirements, court access, disclosure rights, and public-interest audits as routes to greater contestability.

3 BUILDING CAPACITY FOR ALGORITHMIC ACCOUNTABILITY

  • Accountability requires publics to gain power to demand changes and redress within a triadic relationship among actors, forums, and publics.The authors frame accountability as relations in which parties can exert power and make demands upon each other.Source: §2 · p. 5, §3 · pp. 5–6

3.1 On Documentation Practices and Incentivized Ignorance

  • Developer-controlled documentation can enable public demands about deployment, but opacity, selective documentation, and incentives to remain ignorant leave public interests underserved.Regulatory documentation requirements can mediate public access to information and shift organizational practices and incentives toward accountability.Source: §3.1 · pp. 6–7

3.2 Making Algorithmic Harms Cognizable to Courts: The Challenge of Claiming Standing

  • Courts could enforce procedural rights and raise due-diligence standards, yet narrow standing rules often make computational and informational harms difficult to contest or redress.Claims face barriers when harms are not recognized as concrete and imminent injury-in-fact, limiting who can seek redress and what changes can be mandated.Source: §3.2 · pp. 7–9
  • The challenge of standing is not simply court access: plaintiffs’ material claims often go unaddressed, and parties lack effective accountability relationships to understand, contest, or adjudicate harm.Access to courts for computational harms is challenging and is not a panacea.Source: §3.2 · p. 9

3.2.1 Representational Harm

  • In Newman [126], courts dismissed discrimination, speech, and false-statement claims, while missing documentation and expertise prevented adjudication and resulted in preemptive loss of standing.The case involved Black content creators whose videos were restricted and demonetized; the courts’ stated grounds included lack of evidence of intentional discrimination and inability to establish YouTube as a state actor.Source: §3.2.1 · pp. 9–10

3.2.2 Explanatory Harm

  • The GDPR requires firms to provide meaningful information about decision logic [88] and maintain records; ride-hailing workers won disclosure of earnings, assignments, suspensions, and the data and logic behind their use.The authors describe Articles 12–22 as a bulwark against “incentivized ignorance.”Source: §3.2.2 · p. 10

3.2.3 Allocative Harm

  • Allocative harms are difficult to redress when people may not know an algorithm made the decision, its basis is inscrutable, and proving a flawed product or discrimination requires careful expert inspection.The basis for inferences can be opaque even to developers, making it difficult to show that an algorithmic inference caused harm.Source: §3.2.2 · p. 10, §3.2.3 · pp. 10–11
  • In a hiring-system audit, arbitrary changes to a video’s appearance, such as adding a bookcase or changing its tint, significantly changed candidates’ scores [19].The authors note that audits can expose problems but cannot alone establish accountability between parties.Source: §3.2.3 · pp. 10–11

3.3 On Expertise and Consensus in the Adversarial Process of Court Proceedings

  • Disputes over algorithmic methods and general acceptance can threaten expert evidence: prosecutors withdrew ShotSpotter evidence amid controversy over whether it accurately identifies gunshots.The authors note that such a ruling could have set a precedent affecting the tool’s future use in investigations and prosecutions.Source: §3.3 · p. 12
  • In Newman [126], the court rejected Gebru’s expertise as evidence of bias in YouTube’s moderation algorithms because plaintiffs could not show that Google’s and YouTube’s tools were the same.The authors connect this evidentiary gap to YouTube’s lack of obligation to document its moderation algorithms or assess their consequences in a contestable way.Source: §3.3 · p. 12
  • Without accessible documentation and recognized expertise, courts struggle to assess algorithmic harms, while judicial gatekeeping limits what publics can contest and has tended to favor corporate defendants.The authors argue that these barriers also make it harder for publics to cohere around potential harms and demand redress.Source: §3.3 · p. 12

4 IMPACT ASSESSMENTS AND THE LEGIBILITY OF HARMS

Impact assessment procedures can help publics form around shared interests and make algorithmic harms legible to institutions. The authors argue that assessment regimes should structure relationships among developers, government forums, courts, and affected publics—not rely on documentation or audits alone.

4 IMPACT ASSESSMENTS AND THE LEGIBILITY OF HARMS

  • Impact assessments are invoked for diffuse, probabilistic, and potentially novel harms involving intersecting interests.The authors infer from other assessment domains that AIAs have potential to partially address challenges in building algorithmic accountability capacity.Source: §4 · p. 13
  • Assessment procedures may create footholds for publics to seek recognition and redress where courts struggle to recognize computational harms as concrete or particular injuries.The authors argue that procedural rights could protect substantive rights and shift developer incentives toward attending more closely to harms during development.Source: §4 · p. 13
  • The authors argue that assessment regimes should prioritize how accountability relationships are structured over prescribing documentation content.They propose that appropriately designed regimes could make harms more legible to courts and position the public to hold the overseeing forum accountable.Source: §4 · pp. 13–14
  • Environmental assessment litigation shows how procedural rights can help publics cohere and press agencies to consider interests omitted from assessments.Communities used challenges to assessment completeness to establish a proxy substantive right to have environmental-justice interests considered, even without winning every ruling.Source: §4 · pp. 13–14
  • Robust algorithmic impact assessments would require developers to document expected impacts, submit assessments to a government forum, and enable consultation, design changes, and public access to documentation.The forum would mandate consultation with potentially affected stakeholders, require developers to address ameliorable harms, and make aspects of documentation public.Source: §4 · p. 14
  • Audits can expose problems but do not themselves impose legal obligations to compel changes, leaving consequences dependent on firms’ voluntary action.The paper presents adversarial audits as especially important in the absence of robust relations among actors, forums, and publics; assessment regimes aim to create conditions for those relations.Source: §4 · p. 14

5 CONCLUSION

The conclusion argues that accountability regimes should connect developers, regulators, and publics, giving the public ways to organize around shared interests and contest algorithmic impacts. It recommends public access to assessment documentation, researcher access, and feedback between regulators and standards bodies.

5 CONCLUSION

  • Dyadic reporting arrangements risk making regulators dependent on developers to define and measure impacts, while proposed rules offer few clear routes for public contest.The EU proposal offers few opportunities for impacted publics to contest audit terms and outcomes; the Algorithmic Accountability Act underspecifies access and contest mechanisms.Source: §5 · p. 15
  • A robust regime should establish a triadic developer–regulator–public relationship, with public opportunities to cohere around shared interests and contest algorithmic impacts.Court access provides a lower threshold for developer due diligence, but standing is only one part of the broader goal of enabling public demands for change.Source: §5 · p. 15
  • Regulators should build feedback loops with scientific standards bodies, which can receive public and developer input on viable measurement methods.Regulators are rarely equipped to determine developer processes alone.Source: §5 · p. 15
  • The public should be able to review documentation on system purpose and limits, training features, fairness criteria, and real-world performance against developer claims.The conclusion notes that regulations under discussion, including some already deployed, tend to provide little opportunity for public input.Source: §5 · p. 15
  • Regulators should facilitate researchers’ access to impact assessments and aggregated trends to study how the AI/ML industry meets accountability obligations.Source: §5 · p. 15

6 HISTORY DATES

The history-dates passage records a received-date placeholder and notes that review is in progress.

6 HISTORY DATES

  • The recorded history reads “Received [month, year]; [review in progress].”Source: §6 · p. 16
Loading 2203.01455v1…