A Review of Challenges and Opportunities in Machine Learning for Health

• Clinical data support patient-specific treatment decisions and healthcare improvement.Source: Introduction · p. 1
• Machine learning’s data-driven advances and healthcare’s reliance on data motivate machine learning research for healthcare.Source: Introduction · p. 1
• Healthcare machine learning has advanced in diagnosis, pathology, autism subtyping, and large-scale phenotyping.Source: Introduction · p. 1
• Direct application remains fraught because healthcare data are generated and managed primarily to support care rather than machine learning.Source: Introduction · p. 1
• This review focuses on EHRs, emphasizing broad opportunities and careful considerations beyond narrower prior reviews.Source: Introduction · p. 1
• The article covers technical challenges, organizes clinical opportunities into automation, support, and capacity expansion, and highlights shifts, interpretability, and representations.Source: Introduction · p. 1

• Causality, missingness, and outcome definition require careful consideration across supervised, unsupervised, classification, and regression settings.Source: Unique Technical Challenges in… · p. 1
• Causal healthcare questions ask what would happen under treatment interventions, requiring formal causal models beyond classical machine learning.Source: Understanding Causality is Key · p. 2
• Observational data complicate causal learning because actions may reflect an unknown agent policy.Source: Understanding Causality is Key · p. 2
• Simpson’s paradox can reverse observed relationships when additional variables, such as level of care, enter the model.Source: Understanding Causality is Key · p. 2
• Causal models can evaluate treatments and help avoid harmful predictions driven by treatment policies in training data.Source: Understanding Causality is Key · p. 2

Models in Health Must Consider Missingness

• Many healthcare observations are missing even when datasets include all important variables.Source: Models in Health Must Consider… · p. 2
• Missingness mechanisms are classified as MCAR, MAR, or MNAR according to how measurement probability relates to observed or unobserved values.Source: Models in Health Must Consider… · p. 2
• Missingness sources should be examined before deployment because the presence of a laboratory measurement can itself convey patient-state information.Source: Models in Health Must Consider… · p. 2
• Including missingness indicators can improve prediction, whereas ignoring measurement processes can misestimate feature importance and reduce robustness to practice changes.Source: Models in Health Must Consider… · p. 2
• Missingness may reflect differences in access, practice, or recording, producing unfair performance across populations.Source: Models in Health Must Consider… · pp. 2–3

Make Careful Choices in Defining Outcomes

• Reliable outcomes create gold-standard labels for supervised learning and well-defined cohorts for clustering.Source: Make Careful Choices in Defining… · p. 3
• EHR labels should combine heterogeneous sources because structured diagnostic codes may be imprecise or unreliable.Source: Make Careful Choices in Defining… · p. 3
• Phenotyping pools different data types to obtain more reliable labels, including information extracted from clinical notes.Source: Make Careful Choices in Defining… · p. 3
• Predictive performance depends on the criteria underlying medical definitions, which evolve with scientific understanding.Source: Make Careful Choices in Defining… · p. 3
• Clinical actions may be poor labels when observed treatments differ from optimal care or reflect choices other clinicians would make.Source: Make Careful Choices in Defining… · p. 3
• Label leakage can yield high predictive performance without clinical utility when features contain information about the target outcome.Source: Make Careful Choices in Defining… · p. 3

• Healthcare opportunities are organized into automating clinical tasks, providing clinical support, and expanding clinical capacities.Source: Addressing a Hierarchy of Healthcare… · p. 3
• Clinical stakeholders should be engaged early because deployment details can change a technical solution’s intent.Source: Addressing a Hierarchy of Healthcare… · p. 3

Clinical Task Automation

• Well-defined clinician tasks with known inputs and outputs are low-hanging opportunities requiring relatively little domain adaptation and investment.Performance can be measured against existing standards.Source: Clinical Task Automation: Automating… · p. 3
• Automation should optimize clinical workflows rather than replace staff, whose roles may evolve as these techniques improve.Source: Clinical Task Automation: Automating… · p. 3

Automating clinical tasks during diagnosis and treatment

• Medical image evaluation is a natural machine-learning opportunity because clinicians map fixed image inputs to outputs such as diagnoses.Reported applications include diabetic retinopathy, skin-lesion, lymph-node-metastasis, and hip-fracture detection.Source: Clinical Task Automation: Automating… · p. 3
• Routine process automation can reduce staff burden by algorithmically prioritizing emergency triage and summarizing disparate medical-record information.Source: Clinical Task Automation: Automating… · p. 4

Clinical Support and Augmentation

• Clinical support requires identifying pain points with staff and jointly specifying input data, output targets, and evaluation functions.Support addresses work constrained by limited time and resources, which can lead to information loss and errors.Source: Clinical Support and Augmentation:… · p. 4

Standardizing clinical processes

• Standardized order sets and default dosages can support more consistent medication decisions, although static protocols are easier to automate than changing clinical guidelines.Source: Clinical Support and Augmentation:… · p. 4

Integrating fragmented records

• Machine learning can integrate fragmented records to identify patterns such as domestic abuse up to 30 months before healthcare-system recognition.Source: Clinical Support and Augmentation:… · p. 4

Expanding Clinical Capacities

• Digitized records create opportunities for new healthcare capacities, whose impact should be assessed through both innovation and clinical value with substantial clinical collaboration.Source: Expanding Clinical Capacities: New… · p. 4

Expanding the coverage of evidence

• Only 10–20% of treatments were estimated to be backed by a randomized controlled trial, motivating natural experiments to investigate more clinical questions.Trial populations may not represent heterogeneous patients, and individualized care can produce unique treatment pathways.Source: Expanding Clinical Capacities: New… · p. 4

Moving towards continuous behavioral monitoring

• Wearable data can support continuous, non-invasive classifications and alerts, but label leakage, soft labels, confounding, and missingness require careful consideration.For chronic conditions, apparent early detection may identify an existing treatment because patients are often already treated.Source: Expanding Clinical Capacities: New… · p. 4

Precision medicine for early individualized treatment

• Longitudinal data can support individualized treatment effects for syndromes with multiple possible causes, including acute kidney injury.The approach relates to N=1 crossover studies and uses repeated measurements to distinguish causes over time.Source: Expanding Clinical Capacities: New… · p. 5

Opportunities for New Research in Machine Learning

• Promising research directions include handling data non-stationarity, improving model interpretability, and discovering appropriate representations.Source: Opportunities for New Research in… · p. 5

Shift over time

• Changing patient populations and treatment procedures can degrade predictive performance as the statistical properties of the target change.Models should become robust to these changes or acknowledge mis-calibration for the new population.Source: Accommodating Data and Practice… · p. 5
• Google Flu Trends persistently overestimated flu after user search behaviors shifted, motivating models that continuously update.Source: Accommodating Data and Practice… · p. 5

Shift over sources

• Models learned at one hospital may not generalize to another because practices, populations, equipment, and EHR feature mappings differ.Infrastructure for testing across multiple sites and normalization across data-collection practices remains an opportunity.Source: Accommodating Data and Practice… · p. 5

Feedback loops

• Models trained on clinical practice can amplify healthcare biases when deployed predictions influence future training data.The paper connects this risk to feedback loops observed in predictive policing and points to algorithmic fairness research.Source: Accommodating Data and Practice… · p. 5

Creating interpretable models and recommendations

• Black-box models create deployment challenges because clinicians must understand and justify treatment deviations under clinical and legal requirements.The paper notes that models cannot be deployed in clinical settings at low cost.Source: Creating interpretable models and… · p. 5
• Interpretability can be pursued through feature minimization, regularization, model classes with post-hoc analyses, or posterior distributions over decision lists.Decision lists can present relative risks in a form clinicians can use.Source: Creating interpretable models and… · p. 5

Moving from interpretation to justification

• Justifiability should trace the predictive path through individual outputs, training data, or learning algorithms rather than merely highlight features.Influence functions and locally interpretable results are presented as examples, with justification also relevant to security against adversarial attacks.Source: Creating interpretable models and… · p. 6

Adding interaction to machine learning and evaluation

• Collaborative systems can combine the complementary strengths of physicians and learning systems while accounting for clinicians’ caregiving and empathy roles.Source: Creating interpretable models and… · p. 6

Learning meaningful representations for the domain

• Healthcare representation learning should integrate multiple sources and modalities while capturing domain-appropriate structure, hierarchy, similarity, and conditional relationships.These representations should support diverse tasks, multiple valid outputs, domain knowledge, and zero-shot learning in unseen categories.Source: Identifying Representations in a… · p. 6

Conclusion

• Researchers should engage clinical experts early when identifying and tackling important healthcare machine-learning problems.The paper presents collaboration as a route that may lead to clinically useful and operationally feasible models.Source: Conclusion · p. 6