Algorithmic Bias from the Perspectives of Healthcare Professionals

1 INTRODUCTION

• Healthcare AI bias may threaten clinical safety and ethical alignment, including when algorithms lead to Black patients being judged less in need of care.Source: §1 · p. 1
• Bias may arise from training data, population shifts, imbalanced samples, or algorithm design, making its sources difficult to identify.Training data can embed human bias, and models trained in one community may be biased when used with another population.Source: §1 · p. 1
• Assessing algorithms involves trade-offs because performance criteria and ethical principles may conflict, while fairness can be perceived differently across social groups.Algorithm types also differ in design and explainability, from interpretable decision trees to less transparent neural networks.Source: §1 · pp. 1–2
• The study asks how professionals understand bias causes and mitigation, assess algorithms, and intend to use or adopt them.Source: §1 · p. 2
• Healthcare professionals are concerned about algorithmic bias and fairness, and their views on using AI and ML differ across factors.The study gathers quantitative survey data and qualitative comments from professionals in clinical, technical, administrative, IT, and support roles.Source: §1 · p. 2

2.1 Algorithmic Bias

• Algorithmic bias is systematic, unjustified error disadvantaging individuals or groups, rooted in societal biases that can enter technology and amplify discriminatory practices.Source: §2.1 · p. 2
• Healthcare bias can stem from data representation and shift, model choices, or user interaction, while fairness includes several justice dimensions and is studied here as perceived fairness.Performance goals can be difficult to align with all ethical principles.Source: §2.1 · pp. 2–3

2.2 Algorithmic Bias in Healthcare

• Healthcare ML bias has been linked to distribution shifts, underrepresented groups, subjective labels, and automation bias, though some systems have shown fair outcomes across groups or settings.Prior work also identifies concerns about black-box risks and the need for collaborative guidance on data provenance, transparency, and assessment before adoption.Source: §2.2 · pp. 3–4

2.3 Stakeholder Perspectives

• Healthcare stakeholders have different priorities, and prior research reports varied views on ML adoption, bias mitigation, and protected-attribute use.The study uses a framework linking bias to perceived fairness and user acceptance or adoption, with relationships shaped by individual, task, and technology characteristics.Source: §2.3 · p. 4

3 METHODS AND DATA

• The research uses survey and interview methods targeting healthcare professionals in different organizational roles.Source: §3 · p. 4

3.1 Survey

• The survey focuses on supervised classification and uses five-point Likert questions on bias causes, fairness, assessment and tasks, technology characteristics, and intended behavior.Questions drew on healthcare bias literature and technology-acceptance instruments, and were revised through a four-person pilot.Source: §3.1 · pp. 4–5
• Nineteen hospital executive MBA students in a northeastern U.S. healthcare analytics course completed the survey, spanning clinical, administrative, technical, support, and IT roles.Source: §3.1 · p. 5

3.2 Interviews

• Eight medically trained professionals in varied roles and care settings took part in semi-structured interviews lasting 30–45 minutes.Interviews were conducted over Zoom, transcribed, and analyzed using inductive content analysis.Source: §3.2 · p. 5

4.1.1 Causes of algorithmic bias

• Participants strongly agreed that biased training data and cross-hospital distribution shifts can cause bias, but were less certain that larger datasets mitigate it.They also tended to attribute bias to algorithm design.Source: §4.1.1 · p. 5

4.1.2 Fairness of outcomes

• Participants regarded biased outcomes as unfair, rejected excluding protected attributes as a universal remedy, and supported their use depending on the problem.The reported mean for agreement that protected attributes may be used in some applications was 4.4.Source: §4.1.2 · p. 6

4.1.3 Assessment criteria and task characteristics

• Respondents favored multiple assessment criteria and task-dependent prioritization, while views on tolerating bias in high- or low-impact tasks were neutral overall.Clinicians and respondents aged 40–49 were more inclined to reject bias even in low-impact situations.Source: §4.1.3 · p. 6

4.1.4 Technology characteristics

• Participants valued transparency and explainability and preferred them when performance was similar, but were uncertain about accepting lower transparency for substantially better performance.Source: §4.1.4 · p. 6

4.1.5 Intended behavioral responses

• Respondents resisted abandoning algorithms solely for errors, were more inclined to avoid biased or unfair outputs, and would treat algorithmic results as recommendations.Administrators were less likely than other roles to rely on their own knowledge and experience for final decisions.Source: §4.1.5 · p. 6

4.2.1 Causes and consequences of errors and algorithmic bias

• Distribution shift, model bias, and knowledge-base validity were among the concerns raised about automatically generated recommendations.Interviewees noted that risk scores are used alongside other assessments, and that transparency in how scores are generated matters to them.Source: §4.2.1 · p. 7
• A discharge tool’s failure to account for home environments illustrated how regional differences can make a recommendation unsuitable.The example contrasted Florida homes with few or no steps with Massachusetts homes that may have many steps.Source: §4.2.1 · p. 7
• An emergency physician worried that clinicians could become too reliant on AI, while another described using an acuity score as a guide rather than relying on it.Source: §4.2.1 · p. 7

4.2.2 Assessment and response to bias

• Professionals involved in assessment and implementation emphasized working with vendors to understand black-box model parameters and training data.One interviewee favored retrospective testing on the local population and a plan for repeated validation.Source: §4.2.2 · pp. 7–8
• A CMIO described limited physician attention to bias and safety assessment and said this contributed to not implementing many tools that provide clinical recommendations.Source: §4.2.2 · p. 7
• Interviewees said which demographic characteristics merit attention depends on the task, with age relevant in radiology and race and ethnicity warranting closer attention in other settings.One provider expressed optimism that known biases could be corrected automatically.Source: §4.2.2 · pp. 7–8

4.2.3 Technology adoption

• Interviewees wanted transparency about recommendations and training data, but said proprietary vendors do not always disclose training data and clinicians want AI suggestions rather than decisions.They also described working closely with vendors to obtain information while acknowledging that they currently have little of it.Source: §4.2.3 · p. 8

5 DISCUSSION

• Professionals favor treating recommendations as information, with physician judgment deciding; assessment and tolerance of bias vary by task, while transparency preferences and adoption attitudes differ by role.Administrators are less likely to agree to adopt flawed or biased algorithms; practitioners are less concerned about transparency for high-performing algorithms.Source: §5 · pp. 8–9

5.1 Implications for Research and Practice

• The study links algorithmic bias to perceived fairness and intended acceptance or adoption, with opinions varying by roles, perspectives, tasks, and algorithm characteristics.It highlights transparent solutions, communication of uncertainty and fairness threats, and assessment of distribution shift as research priorities.Source: §5.1 · p. 9
• Bias mitigation should account for task and context, assess multiple performance criteria and ethical principles, and prioritize them as needed for the application and stakeholders.The authors describe ensuring fairness and equity in healthcare as a long journey.Source: §5.1 · p. 9

5.2 Limitations

• The study is limited by a small, geographically narrow sample and did not account for the patient populations’ demographic characteristics.A larger sample would also have enabled stratification by participants’ AI/ML knowledge.Source: §5.2 · p. 9

• The research explores healthcare professionals’ opinions about algorithmic bias and fairness issues associated with AI and ML in healthcare.Source: §6 · p. 9
• Future research will extend the inquiry to other healthcare stakeholders, especially patients with varied backgrounds, health conditions, and socioeconomic status.Source: §6 · p. 9
• The paper identifies patient views of healthcare AI and its ethical implications as an open research question.Source: §6 · p. 9

• The survey asks about gender, age, job role, organizational and healthcare experience, algorithm familiarity, AI goals, and general opinions about healthcare AI.Source: APPENDIX: SURVEY QUESTIONS · p. 11
• For this study, “algorithms” means supervised machine-learning techniques producing classification labels, such as positive or negative.Source: APPENDIX: SURVEY QUESTIONS · p. 12
• Questions ask whether bias may arise from training data, cross-hospital use, dataset size, or algorithm design, and whether biased outcomes are unfair or demographic attributes should be used.Source: APPENDIX: SURVEY QUESTIONS · p. 12
• Algorithm-selection questions address multiple assessment criteria, differing priorities by situation, bias tolerance, and transparency or explainability relative to performance.Source: APPENDIX: SURVEY QUESTIONS · p. 12
• Response questions ask whether respondents would avoid algorithms with errors, bias, or unfair allocation, and whether they would use recommendations while retaining final judgment.Source: APPENDIX: SURVEY QUESTIONS · p. 12