Application of machine learning for hematological diagnosis

Study setting and population

• 8,233 first-admission cases from adult patients at UMCL (2005–2015) included 371,341 blood tests, with admission and discharge diagnoses collected.UMCL is a tertiary referral center serving a local population of 400,000; first admissions were used to minimize bias from previous treatment.Source: Study setting and population · p. 2
• 181 blood tests performed at least 10 times were identified after manual curation; 45 parameters (24.9%) were measured in every case.The figure depicts parameter coverage across 43 hematological disease categories.Source: Study setting and population · p. 2, Figure 1 · p. 3
• The reduced set comprised 61 frequently measured basic parameters, excluding diagnosis-confirming tests; 41 parameters (66.4%) were measured in every case.The orange line in Figure 1 delineates the reduced set from the 181 blood test parameters measured.Source: Study setting and population · p. 3, Figure 1 · p. 3
• For diagnostic-performance comparison, the study selected 20 random anonymous adult cases first diagnosed hematologically in 2016 or 2017, comprising 10 women and 10 men.The cases included laboratory blood test results.Source: Study setting and population · p. 3
• Using the UMCL’s modified ICD hierarchy, the models used recorded diseases at three-character depth; 43 distinct hematological disease categories were identified among 8,233 analyzed cases.Source: Study setting and population · p. 3

Predictive model building using a random forest algorithm

• 181 blood-test attributes, 43 disease classes, and fewer than 25% of parameters measured per patient made random forests a logical choice for the study.The data had low noise because blood parameters were determined automatically.Source: Predictive model building using a… · p. 4
• Random forests combine many weak decision or regression trees into a strong learner and can handle missing data, unbalanced data, many attributes, and many classes.They are fast to train and predict, can be parallelized, and generally perform well without parameter tuning, though they may over-fit noisy data.Source: Predictive model building using a… · p. 4
• Early experiments with decision trees, SVM, and Naive Bayesian classifiers yielded significantly lower performance scores than random forests.The passage also notes that random forests performed well in medical diagnostics [15,16] and outperformed hundreds of other machine-learning approaches across many datasets [17].Source: Predictive model building using a… · p. 4

Smart Blood Analytics algorithm

• The pipeline acquires raw database data, filters for tests at treatment start and final diagnoses, preprocesses parameters, builds a model, and evaluates it using stratified 10-fold cross-validation.Preprocessing matches parameters to a reference database, filters erroneous values, and imputes missing values.Source: Smart Blood Analytics algorithm · p. 5
• After successful evaluation, the predictive model is deployed through the Smart Blood Analytics website; otherwise, processing resumes at preprocessing or modeling.The passage identifies Fig. 2 as depicting this process.Source: Smart Blood Analytics algorithm · p. 5, Figure 2 · p. 5

Evaluation of predictive models

• The models were evaluated on all 8,233 cases using stratified tenfold cross-validation repeated 10 times, with one fold held out for testing each time.Folds were selected to keep disease distributions approximately equal.Source: Evaluation of predictive models · p. 5
• Results were compared statistically with the Wilcoxon signed-rank test and summarized in confusion matrices and measures including diagnostic accuracy, specificity, sensitivity, and ROC curves.Source: Evaluation of predictive models · p. 5

ROC curves

• Classification accuracy is valid as an evaluation approach only when all errors have uniform costs.Source: ROC curves · pp. 5–6
• ROC curves were used to examine classifier behavior when false positives and false negatives have different consequences, plotting sensitivity against 1−specificity [18].The passage notes that false positives can incur unnecessary health care costs, while false negatives can delay treatment and endanger patients.Source: ROC curves · p. 6
• For multiclass diagnosis, one-vs-all analysis creates one ROC curve per disease, while two-class scoring classifiers generate curves by varying the decision threshold [19].Source: ROC curves · p. 6
• A disease classifier was considered useful when its entire ROC curve lay above the diagonal, whereas curves near the diagonal indicated performance only marginally better than chance.Source: ROC curves · p. 6
• Overall multiclass performance was assessed with macro-averaging, which weights diseases equally, and micro-averaging, which pools rates across diseases or weights curves by disease frequency [20].Source: ROC curves · p. 6

Clinical test setting

• Clinical testing involved five hematology specialists and eight experienced internal-medicine physicians who were not hematology specialists.The internal-medicine physicians each had at least eight years of experience.Source: Clinical test setting · p. 6
• Each physician assessed 20 cases, naming up to five potential hematological diseases and ranking them by likelihood.Source: Clinical test setting · p. 6
• Both predictive models also made predictions on the same 20 cases used for the physician assessments.Source: Clinical test setting · p. 6

Web-based application

• The application accepts blood parameters and is available to medical professionals upon registration at smartbloodanalytics.com.It also provides a novel visualization of the model’s results.Source: Web-based application and graphical… · p. 6

Graphical representation

• Negative information scores appear inside an offset inner circle, and the chart as a whole is related, though not identical, to Kullback-Leibler21,22 divergence.Positive radii favor a diagnosis, while negative radii indicate that blood-test results speak against it.Source: Web-based application and graphical… · p. 7
• The polar chart shows the 10 most probable diseases: segment angle encodes posttest probability, while radius represents the information score from the posttest-to-pretest probability ratio.Each segment represents one possible disease; the radius is proportional to the logarithm of that ratio.Source: Web-based application and graphical… · p. 6, Figure 3 · p. 7
• The visualization highlights both probable diseases and those with the highest information scores, which physicians should consider together, especially because predicted probabilities may be poorly calibrated.Classifiers may favor more frequent diseases and neglect less frequent ones.Source: Web-based application and graphical… · p. 7

• Both predictive models are available at www.smartbloodanalytics.com upon registration, and Supplementary S2 Data online provides 20 clinical test cases with data and predicted results.Source: Data availability · p. 8
• The study was approved by the Slovenian National Medical Ethics Committee (No. 103/11/15).Source: Data availability · p. 8

Random forest predictive models

• SBA-HEM181 and SBA-HEM061 achieved 59% and 57% accuracy for the top prediction, and 88% and 86% when considering five predictions.Both models were trained across 43 disease categories and validated with stratified tenfold cross-validation.Source: Random forest predictive models · p. 8, Figure 4 · p. 9
• The models’ ROC curves and AUCs were practically identical, and Wilcoxon testing found no significant difference in prediction rankings (p>0.35).SBA-HEM061 performed slightly better for less prevalent diseases, but both models underperformed on rare diseases; the curves almost overlap and AUCs are almost identical.Source: Random forest predictive models · pp. 9–10, Figure 5 · p. 9

Model versus physician comparison

• In 20 cases, top-prediction accuracy was 0.60 (0.55 SBA-HEM61) for the model, 0.62 for hematologists, and 0.26 for internal medicine specialists.For the first five predicted diseases, accuracy was 0.90 (0.85 SBA-HEM61) for the models versus 0.77 for hematologists. Tests found no significant difference from hematologists (p>0.05) and favored the models over internal medicine specialists (p<0.01).Source: Figure 6 · p. 10, Model versus physician comparison · pp. 10–11

Illustrative example

• In an illustrative case, SBA-HEM181 ranked plasmocytoma second using admission results, first using hospitalization results, and fourth using earlier normal tests before subsequent confirmation.The authors suggest the model could help facilitate diagnosis, reduce tests for insidious diseases, and support requests for a second opinion.Source: Illustrative example · p. 11

Discussion

• 0.57 (SBA-HEM61) and 0.59 (SBA-HEM181) classification accuracy for the most probable disease were reported as excellent in a challenging 43-category task, exceeding majority and random prediction.The majority-class prevalence was 0.184 and random prediction accuracy was 0.093.Source: Discussion · pp. 11–12
• In the clinical test, admission-test accuracies of 0.55 (SBA-HEM061) and 0.60 (SBA-HEM181) were not significantly different from the hematologist’s 0.60; top-five accuracy was 0.90 versus 0.77.The top-five difference was significant (p<0.05), while the single-diagnosis comparison was not (p>0.05).Source: Discussion · p. 12
• 57% accuracy on most-probable diagnoses and 90% on discharge diagnoses were reported for SBA-HEM061, trained on one-third of available laboratory variables, suggesting redundancy and interdependency among tests.Its performance was described as equivalent to using all 181 laboratory tests or the 61 routinely used tests.Source: Discussion · p. 12
• The authors propose machine-learning decision support to help non-hematology physicians interpret blood tests, make earlier diagnoses, and refer patients appropriately.They suggest integration into medical information systems or laboratory reports and describe a prospective website test that included a case predicted correctly from tests taken years before symptoms.Source: Discussion · pp. 12–13

Conclusion

• Machine learning recognizes disease-related blood patterns beyond current medical knowledge and can improve diagnostic accuracy over interpretation based on reference ranges.The conclusion says these patterns may include small changes and interactions among multiple blood parameters that are easily overlooked.Source: Conclusion · p. 13
• Small changes and interactions among blood parameters can form disease fingerprints that physicians may overlook, leading to an underestimation of blood-test value.Large changes outside normal ranges are already observed through routine monitoring.Source: Conclusion · p. 13
• Machine learning-based blood diagnosis could change differential diagnosis and modify accepted guidelines, but physicians remain needed to interact with patients and retain access to raw laboratory data.The passage marks the patient-interaction claim with the printed reference marker 4.Source: Conclusion · p. 13
• SBA predictive models indicate substantial information redundancy and unobserved diagnostic potential in laboratory blood test results.The authors describe the models as promising for medical laboratory diagnosis and potentially beneficial to physicians, patients, and healthcare costs.Source: Conclusion · p. 14

• SBA fully funded the research; MarN was its CEO, and MK, MatN, and GG were its advisors.Source: Additional Information · p. 15