PaperlayerView exact revision on arXiv

Source-linked AI summary

Algorithm Design and Physician Liability

Shujie Luan, Shubhranshu Singh, Tinglong Dai

arXiv:2608.13618v1cs.AIecon.TH

TL;DR

Unequal algorithmic performance raises questions about how liability affects both AI design and clinical adoption. The paper models these linked decisions and finds that liability can produce disparate AI use, while equal-accuracy mandates may unintentionally harm both patient groups.

  • Problem

    Existing analysis must account for how rules governing clinical AI use feed back into algorithm design and can reshape adoption and patient outcomes.

  • Method

    The paper uses a two-stage model linking an AI firm’s group-specific accuracy choices to a physician’s subsequent AI-use decision.

  • Results

    Liability tied to disparate AI outcomes can induce disparate use, with physicians consulting AI for a narrower set of disadvantaged patients even at fixed accuracy.

  • Takeaways & Limitations

    Regulating algorithmic disparity can limit AI access for disadvantaged patients, so welfare depends on jointly considering design and clinical-use incentives.

  • Takeaways & Limitations

    The analysis leaves questions about substitution margins for future research.

Abstract

from arXiv · show

A single clinical algorithm can deliver unequal accuracy across patient groups, and concern about such disparity has grown as artificial intelligence (AI) spreads through clinical decision-making. In response, a liability rule introduced in the United States holds healthcare providers responsible when their reliance on disparate algorithms contributes to erroneous clinical decisions. We examine how such liability considerations reshape (i) an AI firm's algorithm design decisions that drive group-specific accuracy and (ii) a physician's decisions to use AI in healthcare delivery. The AI firm designs an algorithm for two patient groups, and improving accuracy for the disadvantaged group is more costly. The physician (who remains the accountable decision-maker) then decides whether to consult AI, weighing the reduction in clinical uncertainty against expected liability exposure when AI errors disproportionately affect the disadvantaged group. We find the liability rule can induce disparate use of AI: the physician may reduce AI use overall and, over an intermediate range of liability, rely on AI less for disadvantaged patients. The effect is non-monotone. As liability increases, the physician's use of AI for disadvantaged patients first declines, then rises as the firm reallocates investment toward reducing disparity or switches to an equal-accuracy design. Mandating equal algorithmic accuracy across patient groups can then inadvertently harm both groups, because a uniform accuracy requirement distorts the firm's investment incentives and the physician's equilibrium AI-use decisions.

1. Introduction

The paper links clinical AI design and physician adoption through deployment-facing liability for disparate algorithmic performance. Liability can reduce AI use for disadvantaged patients at lower levels, but endogenous design responses make its effects non-monotone and equal-accuracy mandates can reduce welfare for both groups.

  • Model: The model jointly links an AI firm’s group-specific accuracy choices with a physician’s decisions to use and follow clinical AI recommendations.Accuracy gains are relatively inexpensive for advantaged patients and costlier for disadvantaged patients; physicians weigh uncertainty reduction against utilization costs and expected legal risk.
  • Results: Liability for disparate AI can induce disparate use: holding accuracy fixed, physicians consult AI for a narrower set of disadvantaged patients.Expected legal exposure acts as an additional shadow cost of reliance, potentially limiting disadvantaged patients’ access to AI.
  • Results: The effect of liability on disadvantaged-patient AI use is non-monotone: use initially declines, then can recover or exceed intermediate-liability levels.At higher liability, the firm reallocates investment toward disadvantaged-group accuracy and may switch to an equal-accuracy design, reducing expected exposure.
  • Results: Mandating equal measured accuracy can reduce welfare for both groups under asymmetric improvement costs.The requirement may substantially lower advantaged-group accuracy while only modestly raising disadvantaged-group accuracy, and reimbursement incentives can increase inappropriate use.
  • Policy implications: Equalizing algorithmic performance and ensuring appropriate clinical reliance are distinct policy objectives requiring complementary instruments.Reimbursement design, utilization guidelines, or monitoring rules may be needed alongside liability standards.

2. Literature

The paper connects liability, medical AI adoption, disparate algorithm performance, and fairness-constrained product design. Its central contribution is showing that deployment-facing regulation can narrow measured accuracy gaps while producing unequal AI use and welfare losses.

  • Liability and expert services: Prior expert-service research shows malpractice liability influences physician decisions and can discipline providers or induce appropriate treatment choices.This literature originates with credence-goods concerns that physicians may overprovide expert services.
  • AI liability: Unlike prior AI-liability work, this paper models regulation that addresses disparate algorithm performance across patient groups and its effects on firms and physicians.The paper focuses on liability from reliance on disparate algorithms rather than liability for disregarding AI recommendations.
  • Physician–AI interaction: The paper links upstream algorithm design with downstream physician adoption, showing AI may be underused under relatively small liability but overused under relatively large liability.Small liability makes liability concerns dominant, whereas large liability strengthens incentives for upstream investment.
  • Disparate algorithm performance: The paper adds that deployment-facing regulation can narrow measured performance gaps yet induce unequal use in equilibrium, so equal algorithmic accuracy need not equalize access.This contrasts with technical approaches that reduce disparate performance by changing objectives, redefining targets, or incorporating group identity.
  • Fairness and product design: Mandating equal accuracy can harm disadvantaged patients even when groups differ only in the cost of improving performance, because a one-size-fits-all constraint distorts physician usage decisions.The welfare loss arises from the physician’s dual objective of representing patients while pursuing AI reimbursement.

3. Model

The model studies treatment selection with two patient types, an imperfect AI signal, and type-dependent algorithmic accuracy. Physician deployment reflects patient outcomes, AI-use costs, revenue, and liability, while a profit-maximizing firm chooses accuracy under asymmetric development costs.

  • Clinical setting: The physician chooses between treatments T1 and T2, with one appropriate treatment yielding benefit b > 0 and the inappropriate treatment yielding 0.
  • Patient types: Patients are type x in an advantaged group or type y in a disadvantaged group, with normalized masses 1 and β, where 0 < β ≤ 1.
  • Information and accuracy: AI accuracy is ρt for patient type t, with 1/2 < ρt < 1; equal accuracy means ρx = ρy, whereas ρx > ρy is disparate for type-y patients.
  • Strategic incentives: The physician retains discretion over AI deployment and weighs patient health outcomes and AI-use costs against revenue and liability, while the firm chooses accuracy to maximize profit.Liability cost ℓ applies when the physician follows a disparate algorithm’s signal for a type-y patient and the resulting treatment is inappropriate; the firm receives payment f per deployment and faces κx < κy.

4. Analysis

The analysis shows that liability changes both physician AI use and the firm’s equilibrium algorithm design. It can reduce AI use for disadvantaged patients, but sufficiently high liability induces an equal-accuracy design that removes the disparity-triggered liability channel.

  • Physician AI use: AI use is optimal only when the signal can change the physician’s treatment decision, and physicians follow the signal whenever they consult AI.For both patient types, consultation has value over an intermediate range of prior beliefs where clinical uncertainty is greatest.
  • Physician AI use: Holding disadvantaged-group accuracy fixed, higher liability contracts the range of priors for which physicians use AI for that group.Liability adds an expected cost to AI use in type-y cases, making adoption and reliance more selective.
  • Physician AI use: Liability generates unequal equilibrium utilization, with lower AI use among disadvantaged patients.Type-y cases activate the liability channel, whereas type-x cases do not, producing asymmetric AI adoption even when the same tool is available.
  • Firm algorithm design: If ℓ<eℓ, the firm develops a disparate algorithm; if ℓ≥eℓ, it develops an equal-accuracy algorithm with ρ∗=1.As liability rises, the firm’s disparate design becomes less profitable and switches to equal accuracy once liability exceeds eℓ.
  • Firm algorithm design: Stronger liability pushes design toward fairness but can reduce aggregate efficiency in accuracy investment.Under disparate design, the firm optimally allocates more performance to type-x patients; strong liability instead induces convergence to equal accuracy.

5. Managerial and Policy Insights

Section 5 shows that liability and equal-accuracy policies can unintentionally worsen AI-use disparities and patient welfare. Their effects depend on physician incentives, reimbursement, uncertainty, and the firm’s accuracy-design response.

  • Liability and AI use: Liability can reduce physicians’ AI use for disadvantaged patients, so the rule may fail to mitigate algorithmic disparities and leave protected patients less likely to benefit.The resulting AI-use disparity can arise even when the policy targets disparities originating in algorithm development.
  • Liability and AI use: AI use for disadvantaged patients is non-monotone in liability: deterrence dominates at low liability, but design improvements or equal accuracy can restore use at higher liability.When the firm supplies an equal-accuracy design, disadvantaged-patient AI use does not depend on liability; with sufficiently strong responses, use can increase monotonically.
  • Physician incentives: Equal accuracy does not ensure appropriate deployment: reimbursement can induce overuse for both groups, while unequal accuracy can support appropriate disadvantaged-patient use when reimbursement offsets expected liability.Equalizing performance removes the group-specific liability channel but not physicians’ private incentive to use rewarded AI.
  • Physician incentives: Physician use can also move from overuse to underuse and back to overuse as liability rises, because reimbursement and liability interact with upstream design responses.At low liability, reimbursement dominates; at intermediate liability, liability suppresses reliance; at sufficiently high liability, equal accuracy relaxes the disadvantaged-group liability channel.
  • Equal-accuracy policy: When liability is small and per-use payment f is intermediate, mandating equal accuracy can reduce aggregate welfare for both patient groups.Because improving disadvantaged-group accuracy is more expensive, parity primarily cuts advantaged-group accuracy while also expanding disadvantaged-group AI use, potentially beyond the clinical margin.

6. Model Extensions

The extensions show that endogenous pricing preserves non-monotone liability effects on AI use, liability is welfare-relevant mainly through design switching, and type-specific priors leave the core mechanisms intact. Equal-accuracy mandates can still reduce welfare for both patient groups.

  • Endogenous pricing: Endogenous per-use pricing makes both price and AI utilization respond non-monotonically to liability.Type-y utilization initially falls as liability rises, then rebounds when stronger accuracy incentives take effect; the fee adjusts in tandem.
  • Endogenous pricing: Type-y AI utilization remains non-monotone within the disparate-design regime even when patients pay the endogenous fee.It decreases below a liability threshold and increases above it, with c = f.
  • Endogenous pricing: Endogenous pricing can sustain service for both patient types and an equal-accuracy outcome, improving firm revenue and AI equity.The firm uses price and accuracy jointly to serve some type-y patients while preserving type-x revenue; equal accuracy also mitigates physician liability concerns and encourages use.
  • Welfare-maximizing liability: Across examined parameterizations, welfare-maximizing liability lies at the boundary where the firm switches between disparate and equal-accuracy designs.Aggregate welfare is defined as total patient health surplus across both groups, not full social welfare.
  • Type-specific priors: With type-specific priors, physicians retain the baseline threshold AI-use rule, while liability can still reduce then increase type-y use and equal-accuracy mandates can harm both groups.The firm still switches from disparate to equal accuracy at a liability cutoff, so allowing αx ≠ αy preserves the use-distortion mechanism.

7. Concluding Remarks

The paper shows that deployment-facing accountability creates feedback between physician AI use and firm algorithm design, so liability can reduce disadvantaged patients’ access to AI before later increasing adoption. Equal-accuracy mandates can also reduce welfare for both groups by distorting accuracy investment and utilization incentives.

  • Core mechanism: Deployment-facing accountability links physician reliance to upstream firm design, so regulating use can reshape design and regulating design can reshape use.The paper models an AI firm and an accountable physician whose decisions jointly determine equilibrium outcomes.
  • Liability and deployment: Liability can reduce AI use for disadvantaged patients, even when physicians continue deploying the same tool for type-x patients.The mechanism is a higher expected cost of relying on AI for type-y patients when unequal performance contributes to inappropriate treatment.
  • Liability and deployment: The liability–use relationship is non-monotone: reliance initially declines for disadvantaged patients, then rises after liability induces greater investment in their accuracy.Thus, one instrument can produce underuse at low liability, a corrective range in between, and renewed overuse at high liability.
  • Equal-accuracy mandates: Equal-accuracy mandates can reduce aggregate welfare for both groups by lowering type-x accuracy and expanding reliance on a still-imperfect tool for type-y patients.Parity is costly to achieve for type-y patients and may be reached largely by reducing type-x accuracy, while equalized performance relaxes the liability deterrent to disadvantaged-group use.
  • Policy implications: Algorithm-performance standards should be paired with instruments governing use, including reimbursement rules, auditing, monitoring, and accountability mechanisms that discipline overuse.Better measured performance need not improve patient welfare when deployment incentives remain misaligned with clinical value.
  • Broader relevance: The mechanism extends beyond healthcare: legal or reputational exposure for group-specific performance differences can distort equilibrium reliance wherever professionals remain accountable for algorithm-informed decisions.The paper identifies analogous settings involving judges, lenders, managers, and employers.

Funding and Competing Interests

The authors report no relevant organizational affiliations or financial or non-financial interests. They also report no funding for the manuscript.

  • All authors certify that they have no affiliations with or involvement in any organization related to the manuscript’s subject matter.
  • The authors report no financial or non-financial interests in the subject matter or materials discussed.
  • The authors have no funding to report.

Appendix

The appendix establishes equilibrium conditions for physician AI use and firm algorithm choice. It shows that liability can generate non-monotone AI use, induce switching between disparate and equal-accuracy designs, and make equal-accuracy mandates harmful for both groups.

  • Physician behavior: Physicians follow the AI signal for type-x and type-y patients whenever they use AI.For both patient types, following and using AI occur in the same cases identified in the respective lemmas.
  • Firm design: No strictly reverse-disparate design can be optimal; when the firm switches designs, the alternative is equal accuracy.The appendix reduces the firm’s relevant comparison to disparate and equal-accuracy designs.
  • Firm design: A cutoff eℓ separates optimal designs: disparate algorithms are optimal for ℓ<eℓ, while equal-accuracy algorithms are optimal for ℓ≥eℓ.The cutoff is unique under the stated feasible-domain conditions; if those conditions fail, equal accuracy never becomes optimal.
  • Physician behavior: Type-y AI use falls when ℓ<¯ℓ and rises when ℓ exceeds ¯ℓ, holding the disparate design fixed.The appendix derives this pattern from the convexity of type-y AI use in liability.
  • Equilibrium implications: The physician underuses AI when ℓs<ℓ<ℓm and overuses AI again when ℓ>ℓm.This establishes the non-monotone equilibrium response to liability around the threshold ℓm.
  • Mandate effects: If fx<f<fy, both patient types are worse off under the equal-accuracy mandate.The appendix states this welfare comparison as Wx(ρm)< and the corresponding type-y inequality.

Online Appendix for “Algorithm Design and Physician Liability”

The appendix extends the paper’s welfare, pricing, liability, and physician-use analyses. It shows that equal-accuracy mandates and liability can generate welfare reversals and non-monotone AI adoption across patient groups.

  • Effect of Mandating Equal Accuracy on Patient Welfare: As f increases, an equal-accuracy mandate typically shifts from harming only type-y patients, to harming both groups, then only type-x patients.Higher f strengthens investment in type-y accuracy under the mandate, eventually reducing the probability that type-y patients are harmed.
  • Effect of Mandating Equal Accuracy on Patient Welfare: As β increases, the mandate similarly shifts from harming only type-y patients, to harming both groups, and eventually primarily type-x patients.A larger type-y market raises the firm’s return to improving type-y accuracy, while a small type-y segment can reduce type-x overuse.
  • Effect of Mandating Equal Accuracy on Patient Welfare: Higher liability can increase the likelihood that type-y patients are harmed by an equal-accuracy requirement.With large ℓ, the disparate equilibrium already induces substantial type-y accuracy investment, leaving smaller incremental gains from the mandate while utilization effects remain.
  • Welfare-Maximizing Liability: The welfare-maximizing liability is attained at a regime boundary: either the largest ℓ preserving the disparate design or the smallest ℓ inducing equal accuracy.This pattern holds across the numerical configurations reported in the main text.
  • Physician AI Use and Liability: As ℓ increases, type-y AI use can first decrease and then increase, because liability exposure initially dominates but induced accuracy gains eventually dominate.The firm may switch from a disparate algorithm below a cutoff to an equal-accuracy algorithm above it.
  • Type-Specific Priors and Comparative Statics: Across a substantial parameter region, mandating equal accuracy harms both patient groups, although small type-y segments or high liability can benefit one group.The mandate can curb type-x overuse when type-y is small and raise inefficiently low type-y use when liability is high.
Loading 2608.13618v1…