Source-linked AI summary
PLeDO: Pain Level Detection for Osteoarthritis from EMR Data
Yuhao Chen, Jiahao Cai, Nafiz Sadman, Farhana Zulkernine, John Queenan, David Barber
TL;DR
OA pain severity is difficult to identify from primary-care EMRs because chart-note expressions are subjective and variable, while structured extraction omits important narrative information. The paper develops SPaDe and the integrated PLeDO tool using NLP, machine learning, chart-note pain expressions, pain scales, and medications; both detect mild and moderate-to-severe pain, though the pilot focuses on two categories because moderate and severe expressions were difficult to separate.
Problem
Primary-care EMRs lack systematic OA pain-level documentation, and existing structured extraction cannot classify pain severity from descriptive chart notes.
Method
The study develops SPaDe to cluster synonymous pain expressions and PLeDO to integrate expression-, pain-scale-, and medication-based information from EMRs.
Results
Both SPaDe and PLeDO detect mild and moderate-to-severe OA pain from EMR data using human-labeled gold-standard data.
Takeaways & Limitations
The study demonstrates the potential of NLP and machine learning to extract OA pain-severity insights from primary-care structured and unstructured clinical data.
Takeaways & Limitations
The pilot focuses on mild versus moderate-to-severe pain because moderate and severe expressions were difficult to distinguish reliably.
Abstract
from arXiv · showhide
Osteoarthritis (OA) is a progressive chronic joint disease resulting in a breakdown of articular cartilage and bone when damaged joint tissues are not able to normally repair themselves. The aim of this pilot research study is to understand the pain severity for OA from patients' primary care Electronic Medical Records (EMR), both from the structured medical data and the unstructured chart note data using information extraction, natural language processing and machine learning techniques. We propose SPaDe, a Synonym-based Pain level Detection tool to categorize patients into having mild or moderate-to-severe pain to understand diagnosis and treatment methods based on only the pain related expressions in the unstructured chart note. Expressions are subjective, objective, and influenced by cultural background and demography which poses a difficult challenge. Therefore, we improve the model by incorporating the medication information from the structured EMR data and pain scale related information from the chart note to propose an integrated pain level detection tool for OA called PLeDO. With the help of human labeled gold standard data, we demonstrate that both SPaDe and PLeDO can detect mild and moderate-to-severe pain from the EMR data to analyze and potentially improve the quality of care in primary care setting.
Introduction
OA pain is clinically consequential but difficult to characterize from primary-care EMRs because chart-note expressions vary and structured extraction omits unstructured notes. The study therefore develops integrated NLP and machine-learning methods to classify OA pain severity from chart notes and medication data.
- Clinical motivation: Canada’s OA rate is projected to increase from 13.8% to 18.6% between 2010 and 2031.
- Clinical motivation: Pain is OA’s most disabling symptom and a major driver of clinical decision making and health-care utilization.The study links pain severity to quality of life, productivity, treatment patterns, and the burden of disease.
- EMR data gap: CPCSSN contains over 2 million patients’ data from 1,500 primary-care clinicians, but its extraction algorithm omits valuable unstructured chart-note data.Manual extraction of free-text clinical data is time-consuming, labor-intensive, and expensive.
- EMR data gap: Canadian EMRs are unlikely to document OA pain systematically, and existing CPCSSN algorithms cannot classify chart-note descriptions into mild or moderate-to-severe pain.
- Study contribution: SPaDe clusters pain-related expressions, while PLeDO combines expression-, pain-scale-, and medication-based information to classify OA pain.Both tools use structured and unstructured EMR information to support pain-level detection in primary care.
Pain Related Classification
Prior pain-classification studies show that clinical text can support automated modeling, but many approaches rely on supervised labels and settings unlike unstructured primary-care OA records. This study addresses that gap with an unsupervised, privacy-constrained framework while confronting substantial variability in clinical notes.
- Prior work: Existing pain-classification research includes numerical-data models and clinical-text NLP systems, but findings remain task-specific.Reported examples include 98% F1 for one numerical-data model and 61% accuracy with 69.5% F1 for cancer-note symptom detection.
- Research gap: Supervised studies commonly depend on manually labeled datasets, disease-specific annotations, or infrastructure supporting large pretrained models.
- Research gap: The study targets OA pain severity in unstructured primary-care EMR notes without gold-standard labels available during model development and within an air-gapped environment.
- Data challenges: Clinical notes contain variable lengths, spelling errors, domain terminology, ill-structured punctuation, numeric and literal expressions, and duplicated longitudinal records.
- Classification policy: The classification policy assigns a patient with multiple pain-level expressions the highest expressed pain level.
Overview of PLeDO
PLeDO classifies OA patients into mild and moderate-to-severe pain groups using structured medications and unstructured chart-note information. The study constructs its sample from Canadian primary-care EMRs, applies exclusions, and develops manually labeled data for algorithm validation.
- Overview of PLeDO: PLeDO integrates pain-expression, pain-scale, and medication-based approaches to categorize OA patients into mild and moderate-to-severe pain groups.
- Overview of PLeDO: The pain-expression system builds three pain-level dictionaries, embeds words as vectors, and compares note words with the dictionaries to classify patients.
- Study sample: The patient sample comes from the OSCAR open-source EMR system and includes primary-care patients diagnosed with OA using the CPCSSN case-definition algorithm.
- Study sample: The initial population contained 23,431 registered patients, including 2,044 patients identified with OA.
- Study sample: After exclusion criteria, the study identified 797 OA patients and obtained deidentified structured and unstructured EMR data in multiple CSV files.Chart notes supplied pain expressions and pain-scale information, while structured data supported gender-based analysis and other study inputs.
- Validation: A randomly selected subset was manually labeled with mild, moderate, and severe pain levels to create gold-standard data for algorithm development and validation.Annotation used predefined protocols developed with medical experts and guidance from an experienced medical-data annotator.
Study Environment
The study uses a secured data-processing environment and an NLP-based pipeline for deidentified EMR chart notes. The workflow covers deidentification, information extraction and cleaning, and prediction-oriented data preparation.
- Study Environment: Deidentified EMR data were staged in the Restricted Data Environment for secure access, analysis, and reporting.
- Study Environment: The first pain-categorization objective used an NLP information-extraction and transformation pipeline to process unstructured EMR chart notes.The stated processing included deidentification, information extraction and cleaning, and development of prediction approaches.
Implementation
PLeDO integrates pain-expression, pain-scale, and medication-based approaches, using custom preprocessing and information extraction to prepare EMR data for pain-level classification. Its workflow includes OA-focused text filtering, linguistic annotation, duplicate removal, regular-expression extraction, and GloVe-based feature representation.
- PLeDO combines pain-expression, pain-scale, and medication-based approaches to categorize OA pain levels.
- Custom cleaning, preprocessing, transformation, and information-extraction algorithms prepare structured and unstructured EMR data for machine-learning analysis.The algorithms account for data structure, layout, context, and representation, then link and transform extracted information.
- OA-related paragraphs are isolated from historical chart notes to reduce information from unrelated diseases.The approach uses keyword search and a comprehensive OA keyword dictionary because copied historical notes contain multiple diseases.
- Duplicate records and formatting artifacts are removed from chart notes and structured medication and demographic data.Cleaning includes removing special characters and punctuation, comparing records, and eliminating duplicated content.
- NLTK NER and POS taggers identify significant terms, including sentiment expressions relevant to pain management.Annotated chart-note words are then extracted for the different pain-level detection approaches.
- Regular expressions extract pain-scale data, while GloVe converts extracted word features into numerical vectors for clustering.Regular expressions are used for specifically formatted data such as dates and pain scales; GloVe captures contextual similarity between synonymous words.
Approach I: Pain Expression-based Approach
The pain expression-based approach uses synonym dictionaries, word embeddings, and similarity-based clustering to categorize OA patients by pain severity. It addresses noisy, subjective chart-note language with unsupervised clustering and patient-level aggregation.
- Motivation: Manual labeling was impractical because notes ranged from 1 to 9,488 words across 482,185 records, motivating unsupervised clustering validated against gold-standard labels.
- Pain-level dictionaries: Three Pain Level Word Dictionaries contain 50 synonyms each for mild, moderate, and severe pain, with automated filtering removing irrelevant or redundant terms.
- Similarity-based extraction: SPaDe embeds dictionary and chart-note words, computes mild, moderate, and severe centroids, and assigns words to the closest centroid using cosine similarity.
- Similarity-based extraction: A similarity threshold of 0.6 balances irrelevant-word noise against excluding relevant wording variations.Thresholds below 0.6 admitted numerous irrelevant words, whereas thresholds above 0.6 excluded relevant variants.
- Patient-level clustering: K-Means clustering addresses the frequency imbalance of mild-pain words and produces interpretable severity-consistent centroids for extracted word vectors.The method avoids letting one potentially misclassified high-severity expression determine the patient label by using the most frequent inferred severity.
- Scope: The proof-of-concept focuses on two categories, mild and moderate-to-severe pain, because moderate and severe expressions were difficult to separate reliably.Even human experts struggled with some moderate-versus-severe chart notes.
Approach II: Pain Scale-based Approach
The pain scale-based approach extracts numerical pain scores from unstructured chart notes and applies a rule-based threshold to categorize patients as mild or moderate-to-severe pain.
- Rule-based categorization: Patients with pain scores exceeding 5 are classified as moderate-to-severe, while all others are assigned to the mild pain group.
Approach III: Medication-based Approach
The medication-based approach maps structured EMR medications and related treatments to mild or moderate-to-severe OA pain categories. It uses analgesic classes and treatment keywords to assign patient-level labels.
- Medication records are consolidated by patient and supplied to the medication-based classification algorithm.The approach combines medication information from multiple records associated with each patient before classification.
- The algorithm extracts prescribed analgesics by matching medication names or chemicals against an analgesic list.Matched medications are stored for subsequent pain-level assignment.
- WHO Analgesic Ladder categories are reduced to mild and moderate-to-severe pain by combining moderate and severe categories.Mild pain corresponds to non-opioid analgesics, while moderate and severe pain involve weak or potent opioids, respectively.
- The treatment keyword dictionary captures OA-related surgery, mobility aids, and pain-clinic referrals as additional moderate-to-severe indicators.Examples include knee or hip surgery, walkers or canes, and pain-clinic information.
- A patient is labeled moderate-to-severe if any medication or treatment keyword matches that category; otherwise, the patient is labeled mild.
Integration
PLeDO integrates expression-based, pain-scale-based, and medication-based approaches to classify OA pain across diverse chart notes. Any moderate-to-severe indication determines the final label; otherwise, the patient is labeled mild.
- Integration: PLeDO combines the outputs of three approaches to generalize across chart notes.
- Integration: The integrated design addresses limited coverage when specific pain tools or scales are unavailable.
- Integration: Any approach indicating moderate-to-severe pain yields that final patient label; otherwise, PLeDO assigns mild pain.
Validation and Results
The section introduces experimental validation of the approaches for categorizing patients into mild and moderate-to-severe pain levels.
- Validation and Results: Experimental validation evaluates the approaches for categorizing patients into mild and moderate-to-severe pain levels.
- Validation and Results: The section discusses results from validating the proposed pain-level categorization approaches.
- Validation and Results: The reported observations and outcomes concern the classification of patients into two pain-level groups.
Validation
The study validated pain-level categorization using manually labeled gold-standard data and evaluated the integrated PLeDO approach through an ablation study. PLeDO combined multiple information sources and achieved the strongest reported overall performance, while the analysis identified important limitations in pain-expression, medication, and chart-note data.
- The gold-standard data validated the clustering, medication-based, and pain-scale-based approaches, while clustering was applied across all 797 data points using pretrained centroids.
- Combining all three approaches produced better results than any single approach, although adding information to SPaDe improved results only a little.
- PLeDO achieved the best accuracy of 0.66 with an F1score of 0.608 and an AU-ROC of 0.621.
- Pain-level detection remained difficult because keyword lists were non-exhaustive, comorbidities introduced noise, and selecting notes containing pain information could introduce bias.
- Removing the pain-scale approach caused the largest accuracy drop, whereas removing SPaDe did not reduce overall accuracy in the combined setting.
- SPaDe had the highest accuracy among individual approaches but lower precision and recall, partly because moderate-to-severe pain was the majority class with 111 of 156 samples.
Conclusion
The study demonstrates the potential of NLP and machine learning to categorize OA pain from primary care EMR data. It proposes SPaDe and an integrated PLeDO framework, while identifying broader clinical NLP, annotation, and data-integration directions for future work.
- SPaDe uses semantic similarity matching and unsupervised clustering to group chart-note pain expressions and notes into mild and moderate-to-severe categories.It uses three pain-level word dictionaries to identify pain-related expressions.
- PLeDO integrates pain-related chart-note information with structured medication data for OA pain-level detection.
- Medication-based and pain-scale approaches provide similar accuracy to the main synonym-based approach.
- Future work will investigate more focused OA populations with fewer comorbidities and advanced clinical NLP tools for improved concept extraction.
- Future research will evaluate supervised and semi-supervised models, add structured clinical variables, explore alternative clustering methods, and investigate privacy-preserving deployable models.The authors specifically mention domain-specific transformer models such as MedicalBERT as candidates for comparison.