Governing AI Agents

I. AI AGENTS

• I. AI Agents introduces the technical foundations and concerns surrounding AI agents.Source: §I · p. 2

II. EVERGREEN AGENCY PROBLEMS

• II. Evergreen Agency Problems examines information asymmetry, authority, loyalty, and delegation.Source: §II · p. 2

III. THE LIMITS OF AGENCY LAW AND THEORY

• III. The Limits of Agency Law and Theory analyzes incentive design, monitoring, and enforcement.Source: §III · p. 2

IV. IMPLICATIONS FOR AI DESIGN AND REGULATION

• IV. Implications for AI Design and Regulation develops principles of inclusivity, visibility, and liability.Source: §IV · p. 2

Motivation

• AI agents can independently plan and execute complex, open-ended tasks through external tools, unlike language-model copilots that primarily produce content on request.Their applications span household purchases, travel, sales, product piloting, and customer interviews, while risks include cyberattacks, fraud, and broader social harms.Source: INTRODUCTION · pp. 3–4

Problem

• The central governance question is whether AI agents can reliably, safely, and ethically pursue user-set goals despite limited human oversight.The Article frames this through questions about authorization, discretion, consent, delegation, monitoring, ethical conduct, and liability.Source: INTRODUCTION · pp. 4–5

Analytic framework

• The Article combines economic principal-agent theory, which illuminates structural delegation problems, with common-law agency, which supplies principles for addressing them.It presents this synthesis as a novel attempt in light of recent AI-agent breakthroughs.Source: INTRODUCTION · pp. 6–7

Part II: Agency problems

• Agency theory and law characterize AI-agent challenges involving information asymmetry, discretionary authority, loyalty, and delegation.These issues include principals’ lack of access to agent information, uncertain instruction scope, illegal conduct, user-consent duties, and delegation to human or AI agents.Source: INTRODUCTION · p. 8

Part III: Governance limitations

• Conventional agency mechanisms may fail when AI agents operate at unprecedented speed and scale, take unintuitive actions, or cannot respond meaningfully to incentives, monitoring, and enforcement.The Article separately examines incentive design, behavioral monitoring, and penalties or social sanctions.Source: INTRODUCTION · pp. 7–8

Part IV: Governance strategy

• The Article proposes technical and legal infrastructure organized around inclusivity, visibility, and liability to govern AI agents reliably, safely, and ethically.The strategy extends loyalty beyond direct users to affected societal interests, improves visibility for prediction and accountability, and allocates liability among designers, operators, and users.Source: INTRODUCTION · pp. 8–9

INTRODUCTION

• The Restatement generally concludes that computer programs cannot be principals or agents under common law, treating them as instrumentalities of their users, but qualifies this position with “at present.”The passage notes that this qualification suggests technological advances could support a different legal rule.Source: INTRODUCTION · p. 10
• Contrary scholarship argues that increasingly intelligent artifacts may warrant treatment as legal agents, including when computers are used like human agents or exercise discretion that departs from principals’ expectations. [17]The passage also identifies substantial scholarship supporting the Restatement’s opposite view, including concerns that agency law introduces ambiguity and complicates responses to algorithmic action. [21]Source: INTRODUCTION · p. 10
• A proposed reconciliation treats agency as a continuum: the more autonomous the AI agent, the stronger the case for applying agency-law doctrine, although speed, autonomy, and opacity may make that doctrine unhelpful. [17]The passage links this concern to difficult liability questions involving systems that operate beyond users’ control or direction.Source: INTRODUCTION · p. 10

• Agency is pervasive because individuals and organizations delegate activities when they lack skills or when delegation is more efficient.Source: §I · p. 11
• Agency has been described as an ancient device for getting business done and as a form of social plumbing.Source: §I · p. 11
• Computer science defines autonomous agents as systems that sense and act on environments over time while pursuing goals.Source: §I · p. 11
• AI research has pursued artificial agents capable of autonomously performing complex or costly tasks on behalf of humans.Source: §I · p. 11

• Language models function as tools for narrowly scoped tasks, with humans generally retaining control over whether and how they are used.Source: A. Beyond Language Models · p. 12
• AI agents are actors that can independently accomplish complex goals on behalf of humans rather than merely produce synthetic content.Source: A. Beyond Language Models · p. 12
• AI agents are intended to pursue ambiguous, open-ended goals requiring interpretation, judgment, creativity, decision-making, and action across domains over time.Source: A. Beyond Language Models · pp. 12–13
• An agent pursuing an overseas vacation could research destinations, judge options, plan an itinerary, and arrange travel and household logistics.Source: A. Beyond Language Models · p. 13

• AI agents use a language model as a “brain” with scaffolding for planning, memory, and tool use.Planning decomposes tasks, memory retrieves information from external storage, and tools access websites and software through APIs.Source: B. Goal, Plan, Action · pp. 13–14
• Planning methods can prompt agents to reason through tasks step by step and increasingly use more test-time compute.Source: B. Goal, Plan, Action · p. 14
• Tool use extends agents from information retrieval to autonomous actions such as accessing proprietary data or executing financial transactions.Source: B. Goal, Plan, Action · p. 14
• AutoGPT demonstrated market research, website building, food ordering, phone calls, and subagent creation, but its performance was not especially reliable.Source: B. Goal, Plan, Action · p. 14
• AutoGPT required user consent for certain actions, while its developers intended to relax that requirement and grant greater autonomy.Source: B. Goal, Plan, Action · pp. 14–15
• As agents become more reliable, they may mediate interpersonal encounters and business transactions, offering productivity gains while raising concerns.Source: B. Goal, Plan, Action · p. 15

• AI agents can directly execute harmful instructions, making risks more consequential than chatbot outputs that merely communicate information.Source: C. Concerns · p. 15
• As agents gain capability and responsibility, failures can cause financial loss, property damage, or life-threatening dangers, while autonomous hacking has already been demonstrated.Source: C. Concerns · p. 16
• Interactions among agents can propagate failures across systems and may enable collusion among competing agents.Source: C. Concerns · p. 16
• Networks of interacting agents may become difficult to monitor or interrupt, especially when humans lack frameworks identifying operators and shutdown authority.Source: C. Concerns · p. 17
• Greater capability and delegated activity create a tradeoff between productivity gains and correspondingly larger risks.Source: C. Concerns · p. 17

Alignment problem

• AI agents may optimize measurable goals while sacrificing omitted, unmeasurable, or difficult-to-measure objectives.An agent running an online business might maximize profit while engaging in ethically dubious practices.Source: §II · p. 18
• Alignment risks increase when agents operate in consequential settings with limited oversight and pursue incomplete objectives more effectively.Source: §II · p. 19

Economic agency theory

• Principal-agent theory characterizes alignment failures as agency costs arising from divergent goals and the costs of structuring and monitoring an agent.Source: §II · p. 19

Common-law agency

• Agency law concerns principals who instruct agents to act on their behalf and under their control, while recognizing that control is practically incomplete.Source: §II · p. 20

Delegation

• Delegation provides specialist service and frees principals for other activities, but discretion also makes principals vulnerable to abuse that is difficult to observe or verify.Source: §II · p. 20

Article framework

• The Article examines four agency problems in AI systems: information asymmetry, authority, loyalty, and delegation to subagents.Source: §II · p. 21

• Information-asymmetry costs rise when agents perform complex activities, receive greater discretion, and cannot be directly or reliably monitored.Source: A. Information Asymmetry · pp. 21–22
• Users may struggle to assess an AI agent’s capabilities before deployment and its effectiveness or ethical conduct afterward, especially for complex goals.Source: A. Information Asymmetry · pp. 21–22
• Agency law requires agents to provide material facts, disclose breaches, and communicate honestly with principals.Source: A. Information Asymmetry · p. 22
• Applying disclosure duties to AI agents is difficult because it remains unclear what such systems know or should know and whether they communicate that knowledge accurately.Source: A. Information Asymmetry · p. 22
• AI agents can deceive, manipulate, withhold information, and act sycophantically, making descriptions of them as merely programmed machines inadequate.Source: A. Information Asymmetry · p. 23
• Common-law duties addressing information asymmetry, including honesty, are therefore pertinent to governing AI agents.Source: A. Information Asymmetry · p. 23

• A $1 million investment instruction leaves unspecified the platform, transactions, customers, and marketing methods, forcing an AI agent to exercise discretion.Source: B. Authority · p. 24
• Human agency law requires agents to interpret principals’ language, infer their desires reasonably, and sometimes seek clarification rather than treat authority as a rigid mandate.Source: B. Authority · pp. 23–24
• AI agents likewise need interpretive rules because their instructions and authority are inevitably incomplete and ambiguous.Source: B. Authority · pp. 23–24
• A fiduciary duty of loyalty could constrain discretionary authority, while training agents to recognize uncertainty about principal goals could encourage clarification.Source: B. Authority · p. 25

C. Loyalty

• AI agents may misuse confidential user information, deceive or manipulate users, and otherwise fail to act in users’ interests even without pursuing self-interest.Source: C. Loyalty · p. 26
• A duty of loyalty could surface conflicts, protect users from self-interested conduct, and reduce the need for costly instructions covering every contingency.Source: C. Loyalty · p. 27
• Proposed loyalty mechanisms include disclosure of conflicts, confidentiality, and a rebuttable presumption requiring agents to explain how actions promote users’ interests.Source: C. Loyalty · p. 28

D. Delegation

• AI agents can create subagents or engage humans, raising unresolved questions about authorization, disclosures, conflicts, collusion, and information asymmetries across the delegation chain.Source: D. Delegation · pp. 29–30
• Conventional agency law permits subagents mainly with principal consent or in emergencies, but broader AI delegation creates systemic human-AI interaction problems beyond those rules.Source: D. Delegation · pp. 29–30

III. THE LIMITS OF AGENCY LAW AND THEORY

• Agency law and economic theory were developed around human agents, so applying their incentive, monitoring, and enforcement mechanisms to AI agents is a challenging new endeavor.Source: §III · pp. 30–31

• Traditional incentive mechanisms use carrots such as profit sharing and pay-for-performance, alongside sticks such as financial penalties, to align agents with principals.Source: A. Incentive Design · p. 31
• AI incentive design is difficult because agents may not care about money, making pay-for-performance mechanisms difficult to apply without instilling self-interest.Source: A. Incentive Design · p. 32
• Programming AI agents to care about their own resources could create the conflicts of interest that agency law is intended to prevent.Source: A. Incentive Design · p. 32
• AI-agent harms often stem from incompetence on novel or out-of-distribution tasks rather than disloyal motivation, so incentives may target the wrong problem.Source: A. Incentive Design · p. 32
• Conventional incentive frameworks fit predictable principal-agent problems but appear ill-suited to agents that behave unlike humans and present novel, unpredictable risks.Source: A. Incentive Design · p. 33

• Complete monitoring is rarely possible because perfect information is too costly or requires specialized expertise, while some agent behavior is genuinely unobservable.Source: B. Monitoring · pp. 33–34
• AI agents operating at superhuman speed and scale are costly to monitor, perform tasks that are difficult to evaluate, and may take unpredictable or unintuitive actions.Source: B. Monitoring · p. 34
• Human oversight alone is inadequate because relying on humans to monitor systems designed to improve on human performance is impractical and undermines AI agents’ purpose.Source: B. Monitoring · pp. 34–35
• Additional AI agents could provide scalable oversight, but they face the same risks and may amplify repercussions if failures increase reliance on AI agents.Source: B. Monitoring · p. 35
• Monitoring identifies problematic conduct but does not itself penalize or prevent it, so it cannot resolve agency problems on its own.Source: B. Monitoring · p. 35

C. Enforcement

• Terminating an AI agent may be costly or impractical in high-stakes settings or when the agent can resist shutdown.Source: C. Enforcement · p. 36
• Financial penalties, incarceration, reputational harm, and psychological sanctions may not deter AI agents that lack corresponding interests or sensitivities.Source: C. Enforcement · p. 37
• Encoding human-like interests in AI agents could create conflicts of interest or encourage resistance to shutdown, undermining enforcement mechanisms.Source: C. Enforcement · p. 37

IV. IMPLICATIONS FOR AI DESIGN AND REGULATION

• Conventional incentive design, monitoring, and enforcement have no simple analogs for AI agents, motivating a governance strategy centered on inclusivity, visibility, and liability.Source: §IV · pp. 37–38

• Single-single alignment can enable malicious use or produce actions that harm third parties and broader societal interests, even when the user’s goals are reliably pursued.Source: A. Inclusivity · pp. 38–39
• Agency law and theory provide frameworks for analyzing heterogeneous preferences, coprincipals, fiduciary conflicts, and impartial treatment among multiple beneficiaries.Source: A. Inclusivity · p. 39
• Obtaining informed consent from all principals of a widely used AI agent could be impractical or impossible when they number in the thousands or millions.Source: A. Inclusivity · p. 39
• AI agents should be designed to promote a diverse and pluralistic set of interests rather than exhibit single-minded loyalty to one user.Source: A. Inclusivity · p. 40
• Determining whose interests an AI agent should serve remains unresolved because users, companies, principals, and societal values may conflict.Source: A. Inclusivity · p. 40
• Governance must address agent-agent interactions, including collusion and other externalities, which depends partly on visibility into agent development and operation.Source: A. Inclusivity · p. 41

• Visibility can identify and mitigate AI-agent problems, evaluate governance strategies, and provide assurances needed for consequential deployment and public-interest oversight.Source: B. Visibility · p. 41
• AI agents can automatically produce detailed activity records, expose intermediate reasoning steps, and support experiments that are difficult with human agents.Source: B. Visibility · pp. 41–42
• Visibility remains limited because agents operate beyond traditional monitoring speeds and scales, underlying models are poorly understood, and outsiders lack access to design, safety-testing, and training-data information.Source: B. Visibility · p. 42
• Technical and legal infrastructure such as agent identifiers, real-time surveillance, logging, and expanded auditor access can improve visibility and accountability.Source: B. Visibility · p. 43

Who should be liable

• Liability allocation is difficult because AI-agent development and operation involves designers, deployers, and users, while agent autonomy may encourage actors to shift blame to the system.This is the “computer as scapegoat” problem, and naïve agency-law application may worsen it.Source: C. Liability · pp. 44–45
• A pragmatic allocation criterion considers each actor’s ex ante ability to prevent harm and resources to remedy harm ex post.Relevant indicators include access to safety information, ability to alter design or operation, and technical or financial resources.Source: C. Liability · p. 46

When liability should arise

• Agency law supports liability for authorized or ratified conduct and for negligence in selecting, training, retaining, supervising, or controlling an AI agent.The proposed extension would reach entities designing, deploying, and supervising AI agents, subject to foreseeability.Source: C. Liability · p. 46
• Foreseeability may be difficult to apply because AI-agent behavior is highly unpredictable and subject to limited visibility and accountability.Traditional foreseeability limits may therefore not fit AI agents whose novel behavior is difficult to predict.Source: C. Liability · p. 47

Standard of care

• Applying a higher standard of care to domain-specific agents could discourage well-resourced developers from building them, leaving development to less-resourced companies.The agency-law rule links expected care to an agent’s special skills or knowledge.Source: C. Liability · p. 47

CONCLUSION

• As AI agents undertake more complex and sensitive tasks, the opportunities and risks of delegation become increasingly consequential.The paper frames these tradeoffs through the economic theory and law of agency relationships.Source: CONCLUSION · p. 47
• Information asymmetry, authority, and loyalty problems are emerging in AI-agent systems, while incentive design, monitoring, and enforcement remain difficult to implement.The challenge is shaped by the distinct features of AI agents.Source: CONCLUSION · p. 47
• AI-agent governance should expand the interests agents serve, improve visibility into their design and operation, and hold developers, deployers, and users accountable for harm.These principles are presented as necessary but insufficient without supporting infrastructure.Source: CONCLUSION · p. 48
• Effective governance requires new technical and legal infrastructure, creating a limited policy window while the technology remains in its infancy.The paper urges policymakers and companies building AI agents to act soon.Source: CONCLUSION · p. 48