PaperlayerView exact revision on arXiv

Source-linked AI summary

Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing

Tal August, Lucy Lu Wang, Jonathan Bragg, Marti A. Hearst, Andrew Head, Kyle Lo

arXiv:2203.00130v1cs.HCcs.CL

TL;DR

Healthcare consumers may turn to medical research when patient-facing resources do not cover their needs, but medical papers can be difficult to navigate and understand. Paper Plain combines question-guided navigation, plain-language summaries, and terminology definitions in an interactive interface. In a usability study, it made reading easier and increased confidence without observable degradation in paper comprehension compared with a typical PDF reader.

  • Problem

    Medical literature provides highly specific information, but healthcare consumers face barriers when interacting with it, including difficulty reading papers and limited coverage in consumer-facing resources.

  • Method

    Paper Plain is an interactive reading interface that combines key questions, paired answer gists, section gists, and definitions of unfamiliar medical terms.

  • Results

    Paper Plain users had an easier time reading research papers and greater confidence that they found relevant information, with similar paper comprehension to a typical PDF reader.

  • Takeaways & Limitations

    Guidance to relevant passages and plain-language summaries alongside original content can make medical papers more approachable and increase readers’ confidence.

  • Takeaways & Limitations

    Medical deployment is limited by the risk of factually incorrect or inconsistent generated text, requiring algorithmic advances or human oversight.

Abstract

from arXiv · show

When seeking information not covered in patient-friendly documents, like medical pamphlets, healthcare consumers may turn to the research literature. Reading medical papers, however, can be a challenging experience. To improve access to medical papers, we introduce a novel interactive interface-Paper Plain-with four features powered by natural language processing: definitions of unfamiliar terms, in-situ plain language section summaries, a collection of key questions that guide readers to answering passages, and plain language summaries of the answering passages. We evaluate Paper Plain, finding that participants who use Paper Plain have an easier time reading and understanding research papers without a loss in paper comprehension compared to those who use a typical PDF reader. Altogether, the study results suggest that guiding readers to relevant passages and providing plain language summaries, or "gists," alongside the original paper content can make reading medical papers easier and give readers more confidence to approach these papers.

1 INTRODUCTION

Healthcare consumers often find medical papers difficult to navigate and understand because of unfamiliar jargon, unfamiliar research conventions, and uncertainty about which information matters. Paper Plain addresses these barriers with an interactive interface that combines guidance, definitions, and plain-language summaries, and its usability study found easier reading and greater confidence without reduced comprehension.

  • The reading problem: Healthcare consumers struggle with medical papers because of unfamiliar jargon, research conventions, and uncertainty about where relevant information appears.Readers may find papers challenging and demoralizing, while spending effort on sections with limited usefulness.
  • Design insight: Paper Plain supplements original paper content with plain-language summaries, called “gists,” that simplify sentences and reduce jargon.Readers can inspect a gist before approaching the corresponding paper content.
  • Paper Plain: Paper Plain combines a key question index, answer gists, section gists, and medical-term definitions to support holistic reading.The key question index guides readers to answering passages and pairs them with plain-language answers; section gists summarize sections in situ.
  • Paper Plain: The first three features—key question index, answer gists, and section gists—are novel for research-paper reading applications, while term definitions provide necessary complementary support.The system integrates novel navigation and summarization affordances with an established terminology feature.
  • Limitations: The authors identify hallucinated, factually incorrect or inconsistent generated text as a limitation requiring algorithmic advances or human oversight.The implementation relies partly on human curation, and medical deployment requires safeguards for incorrect generations.
  • Study findings: Paper Plain users found research papers easier to read and felt more confident they found relevant information, while paper comprehension remained similar to a typical PDF reader.Participants especially favored the key question index and answer gists; they also appreciated section gists and term definitions.

2 BACKGROUND AND RELATED WORK

Healthcare consumers may use medical research for specific, current information, but medical literature remains difficult to navigate and interpret. Prior interactive reading systems address related needs, while Paper Plain focuses on making research papers understandable to healthcare consumers.

  • Online health information can involve barriers including information overload, complex text, unrelated results, and contradictory guidance.
  • Medical literature provides healthcare consumers with highly specific, up-to-date information beyond consumer-facing sites.
  • Readers without research expertise face complex arguments, unfamiliar structures, difficulty judging relevance, and limited domain expertise in scientific texts.
  • Prior systems have supported term definitions, natural-language questions, answer passages, adaptive summarization, concept navigation, and guided reading.
  • Paper Plain extends interactive reading for research papers toward healthcare consumers by combining plain-language support with guidance to relevant information.

3 OBSERVATIONS OF NON-EXPERT READERS

A think-aloud study identified five barriers healthcare consumers encountered when reading medical research papers without prior research expertise. These barriers included understanding terminology and dense text, deciding what to read, finding answers, and relating findings to personal circumstances.

  • The study identified five barriers: unfamiliar terminology, dense text, knowing what to read, searching for answers, and relating findings to personal circumstances.
  • Unfamiliar terminology: Incorrect assumptions about terms integral to an article could lead readers to misunderstand the article.
  • Dense text: Dense sentences, technical terminology, statistics, and complex arguments overwhelmed participants in some passages.
  • Knowing what to read: 11 of 12 participants struggled to judge whether papers contained relevant information and often read exhaustively to determine this.
  • Searching for answers: Participants who knew what information they wanted still struggled to locate details such as patient demographics and previous treatment options.
  • Relating findings to personal circumstances: Some participants wanted patient-specific outcomes, testimonials, or costs that the papers did not provide.

4 PAPER PLAIN: READING SUPPORT FOR MEDICAL RESEARCH PAPERS

Paper Plain targets barriers faced by non-experts reading medical research papers. It combines familiar term definitions with novel navigation and reading support through a Key Question Index and Answer Gists.

  • Paper Plain focuses on helping non-experts know where to invest reading effort in research papers.
  • The interface integrates term definitions with a Key Question Index and Answer Gists to provide navigational guidance and reading support.
  • The design addresses unfamiliar terminology, dense text, knowing what to read, and searching for answers.

Appendix B.

Paper Plain supplements medical papers with on-demand definitions, section summaries, key questions, and answer summaries. In the illustrated reading scenario, these features help a non-expert navigate dense text and identify information relevant to treatment options.

  • Paper Plain’s generated gists supplement rather than replace original paper text because generated language can be vague or inconsistent.
  • Features: Term Definitions provide contextual explanations of unfamiliar terminology through tooltips.
  • Features: Section Gists provide in-situ plain-language summaries that help readers assess dense sections before investing substantial reading effort.
  • Features: The Key Question Index guides readers to relevant answering passages, while Answer Gists summarize those passages alongside the original text.
  • Scenario: In the scenario, the interface helps Sarah identify treatment-related information quickly and feel confident using the approach with future papers.

5 IMPLEMENTATION

Paper Plain combines biomedical question answering and plain-language summarization with term definitions, section gists, key questions, and answer gists. Its prototype uses curated patient-relevant questions, extractive QA, and GPT-3-based simplification, while noting factuality and generation-quality concerns.

  • Paper Plain uses biomedical question answering and plain-language summarization to address barriers to reading medical research papers.
  • Term Definitions: Term Definitions identify medical terms with scispaCy NER and link them to UMLS or Wiktionary definitions.The prototype filters candidate terms by frequency, length, and—in Wiktionary—domain-related tags.
  • Section Gists: Section Gists summarize the first sentence of each paragraph in the lowest-level subsections using GPT-3 prompted for fifth-grade plain language.The authors selected this input strategy to reduce verbatim copying and found the fifth-grade prompt most coherent while retaining some detail.
  • Generation Quality: Generated summaries sometimes contained hallucinated or incorrect information, repetition, or copied source text despite regeneration attempts.The authors regenerated summaries up to five times and selected the most fluent or correct output, but copied text sometimes persisted.

6 USABILITY STUDY

The usability study examined how readers used Paper Plain, how it affected subjective reading experience, and whether it preserved comprehension. Participants read medical papers under Paper Plain variants or a typical PDF-reader baseline in timed, within-subjects scenarios.

  • The study measured feature usage, subjective reading experience, and paper comprehension across Paper Plain conditions.The research questions addressed feature preferences and timing, reading strategies, difficulty, understanding, relevance, and comprehension.
  • Participants: Twenty-four Upwork participants aged 19–67 completed the study, and none had professional medical experience.All participants had completed college; 79% had taken three or fewer STEM courses since high school and 92% had never published a research paper.
  • Procedure: Each participant read papers for 10 minutes after reviewing a diagnosis scenario and MedlinePlus information, then answered paper questions and completed ratings.Participants completed two scenarios involving systemic lupus erythematosus or a herniated disc, with assigned interface–paper configurations balanced across participants.
  • Measures: Comprehension was measured as the proportion of paper-specific multiple-choice questions answered correctly, designed to be clinically relevant and unavailable directly from highlighted passages.Two practicing physicians reviewed the questions, and participants answered seven questions for each of two papers.
  • Conditions: The experiment compared full Paper Plain, guidance-only Questions and Answers, non-guidance Sections and Terms, and a typical PDF reader.The guidance-only variant contained the Key Question Index and Answer Gists; the non-guidance variant contained Section Gists and Term Definitions.

7 RESULTS

Participants used Paper Plain throughout reading and especially favored its guidance features for locating informative passages. They reported easier reading and greater confidence than with a PDF reader, while comprehension was not worse with Paper Plain.

  • Feature Use: All participants with Paper Plain reached the end of a paper, using Section Gists and Term Definitions for dense text and Key Questions and Answer Gists for informative passages.
  • Reading Behavior: 5.19 seconds per paper position with Key Question Index and Answer Gists exceeded the 3.34-second average without them by 1.85 seconds.Readers with the guidance features also jumped back and forth through papers more often, whereas readers without them often read top to bottom.
  • Subjective Experience: Reading difficulty was rated 3.00 with Key Question Index and Answer Gists versus 4.00 with the PDF baseline.The guidance condition also received higher ratings for confidence in finding relevant information and understanding the paper than the PDF baseline.
  • Feature Preferences: 18 out of 20 readers with Key Question Index and Answer Gists selected the Key Question Index as the most helpful feature.
  • Comprehension: Paper Plain comprehension averaged 3.67 correct answers versus 3.50 with the PDF reader, satisfying the reported non-inferiority test.Participants answered 3.73 of 7 comprehension questions correctly on average overall.
  • Gist Use: Participants generally found generated gists useful but often consulted the original text when gists were vague, incomplete, or insufficient for comprehension questions.

8 DISCUSSION & FUTURE WORK

Paper Plain augments medical research papers with interactive guidance and plain-language support, helping healthcare consumers navigate and understand dense content. In a timed usability study, it reduced reported reading difficulty and increased confidence without reducing paper comprehension relative to a typical PDF reader.

  • Paper Plain augments research papers with affordances that help healthcare consumers navigate, evaluate, and understand their contents.
  • Provide reading guidance: The Key Question Index and Answer Gists were the clearest favorite, helping readers quickly jump to informative passages and important sections.
  • 8.1 Summary of the results: Paper Plain significantly lowered reported reading difficulty and increased confidence that participants had found all relevant information.
  • 8.1 Summary of the results: Paper Plain produced similar paper comprehension scores to a typical PDF reader, indicating lower reading barriers without loss in comprehension.
  • Incorporate plain language into the original document: Plain-language features helped participants make sense of papers, but generated text should support rather than replace the original because inconsistencies can misinform readers.
  • 8.4 Limitations: The timed, relatively short 10-minute task may have skewed reading habits, and comprehension differences might become more pronounced with additional time.
  • Improving paper comprehension: Future work should examine Paper Plain in more naturalistic settings and further improve paper comprehension.

9 CONCLUSION

Paper Plain makes medical research papers more approachable by combining plain-language summaries with guidance toward important passages. In a usability study, participants found papers easier to read while maintaining similar comprehension to a typical PDF reader.

  • Paper Plain combines plain-language summaries alongside original content with guidance toward the most important passages.
  • The interface uses natural language processing to provide interactive support for reading medical research papers.
  • Participants using Paper Plain had an easier time reading research papers than participants using a typical PDF reader.
  • Participants most appreciated Paper Plain’s reading guidance through a key question index.
  • Further algorithmic advances are required before Paper Plain can be safely deployed across medical research papers.

A INTERVIEWS WITH HEALTHCARE CONSUMERS AND PROVIDERS

Healthcare consumers and providers described medical research papers as a resource for information beyond patient materials and physician discussions. They sought papers for diagnosis, individualized background, emerging treatments, and treatment comparisons, but faced substantial reading difficulties.

  • Six healthcare consumers and providers were interviewed to understand why people seek medical research papers and what challenges they encounter.
  • The interviews produced scenarios that motivated the interface design and were offered to inspire future research.
  • Participants sought medical literature because they wanted more information than patient-facing resources or discussions with doctors provided.
  • Participants used papers to understand diagnoses in greater depth and find information specific to their circumstances.
  • Participants with chronic or incurable illnesses sought emerging treatment options, including clinical trials and trial results.
  • Participants compared treatments to evaluate doctor-recommended options, clinical-trial results, or alternatives.
  • Reading challenges included unfamiliar terminology, difficulty assessing relevance and trustworthiness, information overload, and exhaustion after one or two hours.

B ITERATIVE DESIGN

Iterative usability evaluations shaped Paper Plain around discoverable guidance, on-demand plain-language support, and preserving access to the original text. Participants reported that prototypes helped them focus on important information and understand papers more broadly.

  • Eight participants evaluated early Paper Plain prototypes in qualitative usability studies.
  • Participants reported that the prototypes helped them access important information, focus attention, understand the paper’s overview, and support their health-information seeking.
  • Provide gists on-demand: Plain-language gists were provided on demand so readers could use supplemental explanations while continuing to engage with dense text.
  • Provide gists on-demand: Readers wanted gists beyond answering passages because restricting simplification to answers limited independent exploration of the paper.
  • Make guidance both discoverable and unobtrusive: The Key Question Index offered an accessible overview, but participants often failed to notice its sidebar until after substantial reading effort.
  • Make guidance both discoverable and unobtrusive: The sidebar could distract readers and obscure PDF features, motivating a toggleable design that balanced guidance with reader autonomy.
  • Supplement, rather than replace, the text: Generated content was designed as a supplement because relying solely on imperfect NLP output could lead readers to misunderstand the paper.

C PAPER PLAIN IMPLEMENTATION

Paper Plain generated plain-language summaries with a GPT-3 prompt adapted for a fifth-grade reading level. The implementation tuned generation length and temperature after comparing outputs across reading-level prompts.

  • The implementation adapted an OpenAI summarization prompt to ask GPT-3 to explain passages in language understandable to a fifth grader.
  • The fifth-grade prompt was selected because second-grade outputs were too general and vague, while later-grade prompts were less consistent.
  • Generation length and temperature were updated as parameters controlling output length and randomness.
  • The final generation settings used a length of 100 characters and a temperature range of 0.25 to 0.5.

D STATISTICAL ANALYSIS

The analysis used linear mixed-effects models to estimate how paper and interface variants related to participants’ measurements while accounting for repeated measurements and individual variation. Coefficients were interpreted relative to the full Paper Plain variant and the herniated disc paper.

  • The researchers fit a linear mixed-effects model for each quantitative measurement, using paper and interface variant as fixed effects.The models were fitted with the lme4 package in R.
  • Random intercepts captured individual variation among participants who contributed repeated, potentially correlated measurements.The participant-specific random intercepts followed a normal distribution with variance σ^2_γ.
  • β0 represented the mean score for Paper Plain while reading the herniated disc paper.
  • The PDF baseline, Term Definitions, and Key Question Index and Answer Gists coefficients represented mean score differences from full Paper Plain under the same paper.
  • The PDF baseline was associated with a 1.9835 higher reading-difficulty score than Paper Plain.The passage identifies this as the same result reported in Table 2.

D.2 F-Tests for Significant Effect of Interface

F-tests found that interface choice significantly affected reading difficulty, relevance, and confidence while controlling for paper and participant-specific effects. Additional ordinal-model checks produced the same conclusions for reading difficulty, confidence, and understanding, although the F-test alone did not identify which interfaces differed.

  • F-Tests for Significant Effect of Interface: p < .001 for reading difficulty, relevance, and confidence in F-tests of interface effects controlling for paper and participant-specific effects.
  • F-Tests for Significant Effect of Interface: p > 0.67, p > 0.98, and p > 0.98 for the interface-by-paper interaction term, so the interaction was omitted from the LMM.
  • F-Tests for Significant Effect of Interface: The F-test identified interface choice as a significant factor across Paper Plain, Questions and Answers, Sections and Terms, and PDF baseline.
  • F-Tests for Significant Effect of Interface: The F-test did not identify which pairs of interfaces differed on each metric.
  • F-Tests for Significant Effect of Interface: Pairwise analyses used two-sided t-tests after the omnibus tests and produced the results shown in Table 2.
  • Ordinal Regression for Likert-Scale Variables: p < .001 for reading difficulty, confidence, and understanding in cumulative link mixed-effects model checks, yielding the same conclusions as the LMM.These models were used because the outcomes were measured on Likert scales.
Loading 2203.00130v1…