Source-linked AI summary
Application of machine learning for hematological diagnosis
Gregor Gunčar, Matjaž Kukar, Mateja Notar, Miran Brvar, Peter Černelč, Manca Notar, Marko Notar
TL;DR
The paper addresses hematologic disease prediction from laboratory blood tests using machine learning. It builds random-forest models with full and reduced parameter sets, finding comparable performance between them and accuracy on par with hematology specialists in clinical testing.
Problem
Rapid, accurate hematologic diagnosis from laboratory blood tests is important, while the usefulness of multianalyte blood-test patterns for disease prediction remains underused.
Method
The study uses random-forest classifiers trained on 181 or 61 blood parameters across 43 disease categories and evaluates them with stratified tenfold cross-validation and a clinical test.
Results
The reduced model performs comparably to the complete model, while both models achieve accuracy on par with hematology specialists and outperform internal medicine specialists in clinical testing.
Takeaways & Limitations
Reduced blood-test parameter sets contain relevant disease information, supporting machine-learning decision support for interpreting hematologic laboratory results.
Takeaways & Limitations
Classification accuracy is not universally valid as an evaluation measure; the authors state that it is valid only under specific conditions.
Abstract
from arXiv · showhide
Quick and accurate medical diagnosis is crucial for the successful treatment of a disease. Using machine learning algorithms, we have built two models to predict a hematologic disease, based on laboratory blood test results. In one predictive model, we used all available blood test parameters and in the other a reduced set, which is usually measured upon patient admittance. Both models produced good results, with a prediction accuracy of 0.88 and 0.86, when considering the list of five most probable diseases, and 0.59 and 0.57, when considering only the most probable disease. Models did not differ significantly from each other, which indicates that a reduced set of parameters contains a relevant fingerprint of a disease, expanding the utility of the model for general practitioner's use and indicating that there is more information in the blood test results than physicians recognize. In the clinical test we showed that the accuracy of our predictive models was on a par with the ability of hematology specialists. Our study is the first to show that a machine learning predictive model based on blood tests alone, can be successfully applied to predict hematologic diseases and could open up unprecedented possibilities in medical diagnosis.
Materials and Methods
The study used anonymized admission and discharge data from 8233 adult hematology cases at UMCL, covering 43 disease categories and 181 blood tests. A reduced set of 61 frequently measured basic parameters was also defined, excluding tests used to confirm specific diagnoses.
- Study population: 8233 adult cases from UMCL admissions between 2005 and 2015 were analyzed using anonymized laboratory results and diagnoses.The dataset included 371341 laboratory blood tests.
- Laboratory parameters: 181 blood tests performed at least 10 times were identified, with an average of 24.9% (45 parameters) measured in every case.
- Reduced parameter set: The reduced parameter set contained 61 of the most frequently measured basic blood parameters and excluded tests used to confirm specific diagnoses.On average, 66.4% (41 parameters) were measured for every case.
- Clinical comparison: An additional clinical comparison used 20 randomly selected anonymous adult cases diagnosed in 2016 or 2017, including 10 women and 10 men.
- Disease categories: 43 hematological disease categories were identified among the analyzed cases using diagnoses coded three characters deep in the ICD hierarchy.
Predictive model building using a random forest algorithm
The study built two Smart Blood Analytics hematology classifiers with random forests, using either 181 blood parameters or a reduced set of 61 parameters across 43 disease categories.
- Random forest method: Random forests combine many small decision or regression trees, allowing weak learners to form a stronger ensemble model.
- Random forest method: The algorithm can handle unbalanced and missing data, large numbers of attributes, and many classification classes, but may over-fit particularly noisy data.
Smart Blood Analytics algorithm
The Smart Blood Analytics pipeline processes laboratory data through acquisition, filtering, preprocessing, modeling, and evaluation before deployment through the SBA website.
- Pipeline stages: The SBA pipeline includes data acquisition, data filtering, preprocessing, predictive modeling, and evaluation.
- Data filtering: Filtering selects blood tests performed at treatment start and uses the final diagnosis as the machine-learning supervisor.
- Data preprocessing: Preprocessing matches blood parameters to a reference database, filters erroneous values, and handles missing values by imputation.
- Deployment: After successful evaluation, the predictive model is deployed through the Smart Blood Analytics website; otherwise, processing returns to preprocessing or modeling.
Evaluation of predictive models
Model performance was evaluated with stratified ten-fold cross-validation and statistical comparison, while ROC analysis addressed classifier behavior beyond accuracy alone. Accuracy-based evaluation assumes that all errors have equal costs, which may not hold in medical diagnosis.
- Cross-validation: Stratified ten-fold cross-validation preserved approximately equal disease distributions across folds while testing each fold against models trained on the other nine.The evaluation process was repeated 10 times.
- Statistical evaluation: Model predictions across 8233 cases were statistically compared using the Wilcoxon signed-rank test and aggregated into confusion matrices.
- ROC curves: Accuracy-based classification evaluation assumes uniform error costs, although false positives and false negatives can have different medical consequences.
- ROC curves: ROC curves were used to visualize classifier behavior in a more general setting than accuracy alone.
Clinical test setting
Five hematology specialists and eight internal medicine specialists independently reviewed laboratory blood-test results from 20 hematological cases and ranked up to five possible diseases.
- 20 hematological cases were assessed by five hematology specialists and eight internal medicine specialists.
- Each physician ranked a maximum of five potential hematological diseases by decreasing likelihood.
- The same 20 cases were also evaluated using both predictive models.
Web-based application and graphical representation of predictive model results
The web application presents model outputs as a polar chart that combines predicted disease probability with information relative to prevalence, helping physicians consider both signals.
- The application displays the 10 most probable diseases as segments of a polar chart.Each segment represents a possible disease.
- Positive radii indicate information favoring a diagnosis, whereas negative radii indicate that blood tests speak against it.Negative information scores are represented inside the graph’s innermost circle because negative radii cannot be depicted directly.
- Segment angle encodes predicted posttest disease probability, while radius reflects the logarithm of posttest-to-pretest probability or the information score.
- Physicians are advised to consider both highly probable diseases and diseases with high information scores because predicted probabilities may be poorly calibrated.The visualization also helps avoid neglecting less prevalent diseases that classifiers may underrepresent.
Data availability
The predictive models and clinical test materials are available online for registered medical professionals, and the study received ethics approval.
- Both predictive models are available at smartbloodanalytics.com upon registration.
- Twenty clinical test cases with their data and predicted results are provided in supplementary online data.
- The study was approved by the Slovenian National Medical Ethics Committee under approval No. 103/11/15.
Results
Both blood-test models performed similarly, with strong top-five accuracy and comparable clinical-test performance to hematology specialists, although rare diseases remained more difficult to predict.
- 88% and 86% were the SBA-HEM181 and SBA-HEM061 accuracies, respectively, when the five most probable diseases were considered.
- Both models underperformed on average for rare diseases, as indicated by the micro-averaged ROC curve lying above the macro-averaged curve.
- The two models had very similar ROC curves and practically identical AUCs, with no significant difference in correct-prediction rankings (p>0.35).
- In a 20-case clinical test, model accuracy was 0.60 and 0.55 for SBA-HEM181 and SBA-HEM61, compared with 0.62 for hematology specialists and 0.26 for non-hematology specialists.
- No significant accuracy difference was found between either model and hematology specialists, while models performed better than internal medicine specialists (p<0.01).
- The plasmocytoma case illustrates how the model could facilitate diagnosis, reduce tests in insidious disease, and support requests for a second opinion.
Discussion
The study reports that random-forest models using multianalyte blood-test results predicted 43 hematologic disease categories at specialist-comparable performance, while supporting clinical decision-making.
- 43 disease categories made this a comparatively difficult medical classification task, unlike datasets focused on binary disease presence or absence.
- 0.57 and 0.59 accuracy were achieved for the most probable disease, exceeding majority and random baselines of 0.184 and 0.093.
- Machine-learning models could help physicians interpret multianalyte blood-test results, addressing reported difficulty in ordering and understanding laboratory tests.
Conclusion
The conclusion emphasizes that blood parameters contain underused diagnostic information, including subtle and interacting patterns, and that machine-learning models show promise for laboratory-based diagnosis.
- Small changes and interactions among multiple blood parameters can form disease fingerprints that are easily overlooked when physicians focus on values outside reference ranges.
- Machine learning can recognize disease-related blood laboratory patterns beyond current medical knowledge, according to the authors.
- Sufficiently large datasets indicate substantial information redundancy and unobserved potential in laboratory blood-test results for diagnosing disease.
- The models show promise for medical laboratory diagnosis and could be valuable to physicians and patients, with beneficial impacts on healthcare costs.
Additional Information
The research was fully funded by Smart Blood Analytics Swiss SA, whose CEO and advisors are listed among the authors’ financial interests.
- Smart Blood Analytics Swiss SA fully funded the research, and several named authors held company roles as CEO or advisors.