Source-linked AI summary

An AI-Resilient Text Rendering Technique for Reading and Skimming Documents

Ziwei Gu, Ian Arawjo, Kenneth Li, Jonathan K. Kummerfeld, Elena L. Glassman

arXiv:2401.10873v1cs.HC

TL;DR

Readers need help consuming difficult text, but summarization can omit, hallucinate, or misrepresent information in ways that are hard to notice. GP-TSM preserves the original text while using recursive sentence compression to render successive levels of detail in lighter gray. In user studies, it improved reading-related measures and was preferred over a non-grammaticality-enforcing alternative and prior word-level rendering methods.

  • Problem

    Summarization can omit, hallucinate, or misrepresent information that readers may not detect without rereading the original document.

    Source: §1 · p. 2
  • Method

    GP-TSM uses recursive sentence compression to identify detail beyond a passage’s core meaning and renders that detail in successively lighter but legible text.

    Source: §1 · p. 3, §3.2.1 · p. 8, §3.4 · p. 10
  • Results

    User studies suggest GP-TSM enhances reading efficiency and overall user experience, reduces perceived reading difficulty, and outperforms its non-grammaticality-enforcing twin, NGP-TSM.

    Source: §5 · p. 22
  • Takeaways & Limitations

    GP-TSM offers an AI-resilient way to support reading and skimming by changing visual saliency while retaining the original document.

    Source: §1 · p. 3, §5 · p. 22

    LLM-driven decisions are black-box and computationally complex, reducing transparency and potentially slowing real-time deployment.

    Source: §6 · p. 23

Abstract

from arXiv · show

Readers find text difficult to consume for many reasons. Summarization can address some of these difficulties, but introduce others, such as omitting, misrepresenting, or hallucinating information, which can be hard for a reader to notice. One approach to addressing this problem is to instead modify how the original text is rendered to make important information more salient. We introduce Grammar-Preserving Text Saliency Modulation (GP-TSM), a text rendering method with a novel means of identifying what to de-emphasize. Specifically, GP-TSM uses a recursive sentence compression method to identify successive levels of detail beyond the core meaning of a passage, which are de-emphasized by rendering words in successively lighter but still legible gray text. In a lab study (n=18), participants preferred GP-TSM over pre-existing word-level text rendering methods and were able to answer GRE reading comprehension questions more efficiently.

1 INTRODUCTION

Readers struggle with long, complex, or otherwise difficult text, while summarization can omit, hallucinate, or misrepresent information that readers may not detect. GP-TSM instead preserves the original document while visually de-emphasizing less central detail through recursive sentence compression, and it improved efficiency and preference in user studies.

1 INTRODUCTION

  • Unlike summarization, which produces a lossy representation, GP-TSM modifies the visual attributes of the original text to support faster reading and skimming.The approach aims to retain the original content while making important information more salient.Source: §1 · pp. 2–3
  • Summaries can omit details important to a reader’s context, introduce false information, or shift meaning through selection and paraphrase.These errors include omission, hallucination or confabulation [45] [66] [107], and misrepresentation.Source: §1 · p. 2
  • Personalized summarization remains limited because readers’ interests, contexts, and tasks can be unobservable in the moment.It also relies on coarse characterizations of users.Source: §1 · p. 2
  • Readers may be unable to detect these summary errors without rereading relevant portions of the original document.Omissions leave no visible trace in the summary, while hallucinations can appear plausible and misrepresentations require comparison with the source.Source: §1 · p. 2
  • GP-TSM recursively compresses sentences to identify successive detail levels, rendering earlier-removed words in progressively lighter but still legible gray.Words cut earlier in the recursive compression process receive lighter text, while the reader can choose a minimum opacity level.Source: §1 · p. 3
  • The method preserves grammatical subsets of each sentence at chosen opacity levels, supporting a more natural reading flow.This grammar-preserving design was supported by a formative study using a semi-automated sentence-compression backend.Source: §1 · p. 3
  • GP-TSM helped readers complete non-trivial GRE reading comprehension tasks more efficiently and was strongly preferred over unigram-frequency font-opacity rendering.The final within-subjects study compared GP-TSM with WF-TSM, a control based on unigram frequency.Source: §1 · p. 3

2 RELATED WORK

This section situates GP-TSM within research on reading and skimming, summarization, and text-rendering modulation. It contrasts GP-TSM’s recursive, grammar-preserving approach to identifying nested subsets for de-emphasis with prior text-saliency methods and discusses its versatile visual attribute modulation.

2.1 Reading and Skimming

  • Skimming, especially in unfamiliar contexts, trades reading speed for comprehension and can cause readers to miss key information, while long, complicated sentences can be hard to understand.At 43 words, comprehension drops to less than 10%.Source: §2.1 · pp. 3–4

2.2 Text Summarization Methods

  • Extractive summarization preserves source wording, whereas abstractive summarization is more flexible but both approaches can alter semantic meaning.GP-TSM instead uses recursive sentence compression to create nested detail levels while keeping all original text available for recovery from AI errors.Source: §2.2 · pp. 4–5

2.3 Text Rendering Modulation Methods

  • Text-rendering research modulates fonts, opacity, size, color, and other visual attributes to improve readability, guide attention, or support skimming.Prior systems include FontMART, QuickSkim, word-level typography methods, and sentence-level color or saliency encodings.Source: §2.3 · p. 5
  • GP-TSM extends existing text-rendering modulation by focusing on making key information more salient for focused reading and efficient skimming.Its primary contribution lies within the established tradition of visual text modulation rather than replacing the original document with a summary.Source: §2.3 · pp. 5–6
  • GP-TSM recursively compresses sentences to distinguish multiple nested grammatical subsets, enabling a new scope of visual attribute modulation.This computation identifies what to de-emphasize rather than relying only on word- or sentence-level saliency.Source: §2.3 · p. 6
  • Word-frequency methods highlight unique words but ignore document meaning and within-sentence relationships that GP-TSM considers.These methods weight words by English-language or document-level unigram frequency.Source: §2.3 · p. 6
  • Like HiText, GP-TSM uses an AI model, but it performs recursive sentence compression rather than ranking sentences by predicted importance.HiText uses a neural-network-based model to rank sentences according to predicted importance.Source: §2.3 · p. 6
  • GP-TSM is versatile and easily integrable with systems that use highlights, annotations, colors, opacity, or saturation to visualize text analyses.Examples include Scim [34], VarifocalReader, and TextViewer.Source: §2.3 · p. 6

3 GP-TSM

This section presents GP-TSM, a text-rendering method designed to reduce reading and skimming demands while preserving original content across multiple detail levels. It covers the method’s interaction design and paragraph-level computation, including graded opacity, grammatical subsets, and an LLM-based approach to recursive summaries.

3.1 Design Goals

  • GP-TSM aims to preserve original wording while integrating into existing reading workflows, supporting multiple detail levels, coherent skimming, and recovery from AI decisions readers disagree with.Lower-detail text remains available rather than hidden, so readers can notice and judge automated choices without extra actions.Source: §3.1 · pp. 6–7

3.2 Design Process

  • GP-TSM emerged from iterative browser prototypes spanning text attributes, modulation scopes, interaction techniques, computation scopes, and rendering methods.The design space was organized around dimensions and alternatives summarized in Table 1, followed by prototypes exploring combinations without implementing every feature combination.Source: §3.2 · p. 7

3.2.1 Text Attribute to Modify.

  • GP-TSM modulates opacity because fading provides graded, visually cohesive emphasis that distinguishes detail levels while remaining less distracting than bold or other font changes.For black text on white, lighter text signals detail and darker text signals criticality; the least opaque text remains legible under WCAG contrast guidance.Source: §3.2.1 · pp. 7–8

3.2.2 Scope of Attribute Modification.

  • GP-TSM applies opacity to grammatical subsets within sentences, revealing successively smaller detail levels through recursive sentence compression rather than independently styling words or phrases.This preserves coherent sentences when readers skim at a chosen opacity level, while retaining as much original semantic meaning as possible.Source: §3.2.2 · pp. 8–9

3.2.3 Interaction techniques, if any.

  • GP-TSM targets desktop reading because larger screens can display more information, and it focuses on mouse-and-keyboard interaction rather than mobile gestures.The authors chose desktop interactions for longer, complex documents based on reported reading benefits and the ability to avoid frequent scrolling or zooming.Source: §3.2.3 · p. 9
  • After piloting sliders, carousels, and scrolling, the final interface lets readers toggle GP-TSM on or off by keyboard or mouse while saliency is determined automatically.Hiding information and interactive controls could disrupt comprehension, attention, workflow integration, and recovery from automated decisions, motivating a passive display.Source: §3.2.3 · p. 9
  • GP-TSM is intended to reduce the cognitive load required for reading, an already demanding task.Source: §3.2.3 · p. 9

3.2.4 Scope of Computation.

  • Paragraphs provide enough text for substantial de-emphasis while retaining a coherent theme for summarization, so GP-TSM uses them rather than sentences or documents.Sentence-level processing de-emphasized little text, whereas document-level choices risked retaining ideas a reader might not consider most critical.Source: §3.2.4 · p. 9
  • GP-TSM lets readers notice and recover from differences between their judgments and the AI’s without additional action, but larger-scale errors may prompt them to turn it off.The method does not prevent annoyance when the AI makes choices users consider wrong.Source: §3.2.4 · p. 9

3.2.5 Extractive summarization method.

  • Classical syntax-based methods produced ungrammatical or incomplete outputs, often removed important adjectives and noun phrases, and existing extractive methods could not generate multi-level recursive summaries.Source: §3.2.5 · p. 10

3.3 Overview of the GP-TSM System

GP-TSM renders text at multiple opacity levels based on successive recursive shortenings that preserve grammaticality. Words removed earlier appear lighter, while words retained across all levels remain fully salient.

  • GP-TSM produces multiple nested levels of grammatically correct detail by repeatedly shortening each sentence and paragraph.Source: §3.3 · p. 10
  • Words removed in the first shortening round receive the least opacity, while words removed later appear progressively more opaque.Source: §3.3 · p. 10
  • Words that survive every shortening round remain in full color, identifying the text retained in the most concise extractive summary.Source: §3.3 · p. 10
  • Reduced opacity marks content judged less relevant to a sentence’s core meaning, whereas simple sentences retain salience.Source: §3.3 · p. 10

3.4 Algorithmic Workflow

The workflow uses an LLM to generate recursive extractive compressions while enforcing verbatim preservation, grammaticality, and quality selection. It then maps deletion order to opacity levels and stops when further deletion is unavailable.

3.4 Algorithmic Workflow

  • The workflow requests deletion-only edits so the rendered passage remains the original paragraph rather than an LLM rewrite.Source: §3.4 · pp. 10–11
  • The evaluator averages semantic fidelity, target length, paraphrasing, and grammatical correctness to select among candidate compressions.Source: §3.4 · pp. 11–12
  • GP-TSM repeatedly compresses each paragraph to identify multiple levels of sentence-level criticality, using each selected output as the next round’s input.Source: §3.4 · p. 12
  • Each round requests eight GPT-4 responses, reverts additions and substitutions, evaluates quality, and retains the highest-scoring shortened paragraph.Source: §3.4 · pp. 11–12
  • The process stops when none of the eight responses deletes additional words, allowing complex paragraphs more levels than simpler paragraphs.Source: §3.4 · p. 12

3.4.1 Implementation.

  • The implementation is a Flask application configured to call OpenAI’s GPT-4 model, while the authors note that LLM use is nondeterministic.Source: §3.4 · p. 10, §3.4.1 · p. 12

4 USER STUDIES

GP-TSM was evaluated in a preliminary study of a partially automated visualization and a summative study of the fully automated system. Interfaces were presented under neutral color labels during the studies.

  • The evaluation comprised a preliminary user study of visualization effectiveness and a summative study measuring the fully automated GP-TSM’s impact.Source: §4 · p. 12
  • The supplied study description establishes the evaluation structure but does not report participant outcomes or comparative performance.Source: §4 · p. 12
  • Every tested interface received an arbitrary color label such as “reader-green” or “reader-blue” during the user studies.Source: §4 · p. 12

4.1 Preliminary User Study

A preliminary within-subjects study with 18 participants evaluated human-in-the-loop GP-TSM against plain text and an interactive variant using GRE reading tasks. It examined comprehension, completion time, and reading experience, finding benefits for HITL-GP-TSM while interactive granularity provided no significant advantage and was not retained.

4.1.1 Overview.

  • The study used 18 participants and compared human-in-the-loop GP-TSM, an interactive granularity variant, and plain-text Control in a within-subjects design.The human-supervised backend isolated the visualization’s efficacy from fully automated backend quality; all conditions used Lato at 14pt.Source: §4.1.1 · pp. 12–13
  • The interactive variant let readers hide text below an opacity threshold, but this could obscure automated decisions and prompted participants to explore its controls.The authors therefore treated interactive granularity’s value as less clear and did not include this variant in the later fully automated evaluation.Source: §4.1.1 · p. 13

4.1.2 Procedure, Participants, Conditions, and Measures.

  • Participants completed three counterbalanced GRE long-passage tasks, each with four questions, under separate interface conditions and within a 60-minute study session.Each task allowed up to 10 minutes, and post-task questionnaires assessed experience, difficulty, workload, and perceived performance.Source: §4.1.2 · p. 13

4.1.3 Results.

  • Interactive GP-TSM participants answered approximately three-fourths of an additional GRE question correctly out of four versus Control, while static GP-TSM gained approximately half.The condition effect was significant (p=.03, F2,11.3=4.81).Source: §4.1.3 · pp. 14–15, Figure 4 · p. 15
  • 7.9min was the average completion time with static GP-TSM versus 9.3min with Control, a significant 1.4-minute reduction; other tests were nonsignificant.The condition effect was significant (p=.022, F2,10.6=5.52), while interactive GP-TSM was not significantly faster than Control.Source: Figure 4 · p. 15, §4.1.3 · p. 15
  • Static GP-TSM received significantly more positive ratings than Control in 5 of 9 shared questions, whereas interactive GP-TSM differed only on overall experience.Participants generally preferred both GP-TSM conditions over Control.Source: §4.1.3 · p. 15
  • HITL-GP-TSM produced significantly better performance and reading experience than Control.Table 2 reports post-task survey scores on 7-point Likert scales, with significance assessed using ANOVA and Tukey’s HSD tests.Source: §4.1.3 · p. 15, Table 2 · p. 16
  • The difference between HITL-GP-TSM-Interactive and HITL-GP-TSM was not significant.Source: §4.1.3 · p. 15
  • HITL-GP-TSM alone achieved both faster task completion and significantly lower perceived difficulty, so interactive granularity was dropped from further development.Source: §4.1.3 · pp. 15–16

4.2 Main User Study of the fully automated GP-TSM

This section examines a fully automated user study of GP-TSM with 18 participants, focusing on reading performance, preferences, and participants’ experiences with its multilevel visualization. It also considers qualitative feedback on comprehension, reading efficiency, grammar preservation, and needs for guidance, legibility, and customization.

4.2.1 Overview.

  • The study used a separate set of 18 participants, replaced the interactive granularity condition with WF-TSM, and added a component comparing GP-TSM with NGP-TSM.NGP-TSM removed grammaticality constraints from both the LLM prompt and response evaluator.Source: §4.2.1 · pp. 16–17

4.2.3 Quantitative Results.

  • GP-TSM produced significantly higher comprehension scores and faster task completion than the control conditions, outperforming Control and WF-TSM overall in the user study.Comprehension showed a significant condition effect (p=.02, F2,11.3=4.79), and GP-TSM participants scored significantly higher than Control participants (p=.021).Source: Figure 6 · p. 18, §4.2.3 · p. 18
  • Participants generally preferred GP-TSM, rating it more positively than Control on 4 of 9 shared questions and than WF-TSM on 6 of 9.For opacity-specific questions, GP-TSM exceeded WF-TSM on all four questions, particularly regarding intelligibility and helpfulness for seeing relationships within sentences.Source: §4.2.3 · pp. 18–19
  • Future-use ratings favored GP-TSM, with means of 5.8 (SD=1.8), 4.7 (SD=1.3) for Control, and 3.3 (SD=2) for WF-TSM (p<0.05).Participants also strongly preferred GP-TSM when directly ranking the conditions for reading comprehension helpfulness.Source: §4.2.3 · p. 19, Figure 7 · p. 20

4.2.4 Qualitative Feedback.

  • Twelve of 18 participants said GP-TSM facilitated more efficient, less congested reading by letting them skip grayed-out words while retaining the passage’s gist.Participants described a two-step process: first skimming black words for the gist, then returning to selected sections for more detail.Source: §4.2.4 · p. 20
  • Participants said GP-TSM’s visualization helped them focus on key points and read more efficiently, with graying perceived as natural, consistent, and appropriately contrasted.Some participants also expressed interest in future use, including as a Chrome extension.Source: §4.2.4 · pp. 20–21
  • Participants requested clearer explanations or tutorials because the meaning of the gray levels was not always immediately intuitive.This recurring feedback suggests that the interface’s visual encoding needed more explicit guidance.Source: §4.2.4 · p. 21
  • Participants also requested darker or differently colored text, larger fonts, and controls to customize or hide grayed words.Some found the lightest gray difficult to read, while others wanted adjustable graying or the ability to show only black text.Source: §4.2.4 · p. 21

4.2.5 GP-TSM vs. NGP-TSM: the Comparison Interview.

  • 15 of 18 participants perceived a difference between GP-TSM and NGP-TSM, while 9 of 18 identified grammaticality as the key difference.Some participants sensed a difference without specifying it, whereas others noticed that NGP-TSM grayed structurally important articles, prepositions, and determiners.Source: §4.2.5 · p. 21
  • All 15 participants who perceived the difference preferred GP-TSM to NGP-TSM for better comprehension and reading efficiency.The study attributes these benefits to preserving grammar at every level, allowing readers to skip gray text without losing meaning.Source: §4.2.5 · p. 21

5 DISCUSSION

GP-TSM was positively evaluated as a grammar-preserving visualization that supports efficient reading and layered engagement while retaining the original text. Participants generally preferred its saliency choices over WF-TSM, though opacity levels could be difficult to distinguish or read.

5 DISCUSSION

  • Participants preferred GP-TSM’s choices about which text to gray out over WF-TSM, while still noticing and recovering from automated judgments.Participants could identify transition words that the backend undervalued, illustrating the intended AI-resilience property.Source: §5 · p. 22
  • GP-TSM’s grammar-preserving design was clearly perceived and praised when participants compared it with the non-grammatical NGP-TSM variant.Participants responded positively to the visual text attribute being modulated, namely opacity.Source: §5 · p. 22
  • GP-TSM enhanced reading efficiency, overall user experience, and perceived reading difficulty across multiple measures.Performance gains and reduced task time provided especially compelling quantitative evidence relative to controls.Source: §5 · p. 22
  • GP-TSM’s opacity-based rendering preserves the original text and presents multiple grammatical detail levels, letting readers choose their engagement depth.Successively lighter text provides contextual cuing while grammatical preservation supports coherent reading and skimming.Source: §5 · p. 22
  • Opacity-level design involved a tradeoff between keeping the lightest text legible and making levels sufficiently distinct for consistent selection.Differences became harder to perceive in complex sentences with many levels, and some participants struggled with light gray text.Source: §5 · p. 22
  • Participants described a two-step reading pattern: first extracting the darker text’s overview, then revisiting lighter text for details prompted by interest or questions.This layered approach may help explain the observed improvements in reading efficiency and comprehension.Source: §5 · pp. 22–23

6 LIMITATIONS AND FUTURE WORK

The discussion identifies limitations involving the LLM backend, participant sample, text generalizability, measurement, deployment speed, and accessibility. Future work should test broader users and domains while improving transparency, real-time feasibility, and evidence about reading behavior.

6 LIMITATIONS AND FUTURE WORK

  • The LLM backend reduces transparency, may vary across similar inputs, and can produce heuristic selections that miss the most contextually relevant response.The authors retain an LLM-based approach because it currently produces the best paragraph-level saliency modulation quality and relevance.Source: §6 · p. 23
  • LLM processing complexity slows the system and may hinder deployment in real-time reading scenarios.The authors expect improvements in LLM quality and speed to make uses such as a Chrome extension more feasible.Source: §6 · p. 23
  • The limited sample size and sampling procedure may have skewed conclusions by underrepresenting diverse participant backgrounds.Future evaluations should include varied ages, educational backgrounds, and cultural and linguistic contexts.Source: §6 · p. 23
  • GP-TSM’s effectiveness remains uncertain across other text styles and formats because comprehension was evaluated using long GRE passages.The authors specifically identify technical manuals, legal documents, and everyday communication as untested domains.Source: §6 · p. 23
  • Self-reported interaction accounts may be biased, motivating future eye-tracking studies of how GP-TSM shapes reading and skimming.The authors distinguish these reports from more direct behavioral evidence that future work could collect.Source: §6 · p. 23
  • Font-opacity modulation can reduce legibility and accessibility, especially for people with visual impairments, despite following WCAG contrast-ratio guidelines.The authors also call for evaluations involving people with cognitive impairments.Source: §6 · p. 23
  • GP-TSM’s AI-resilience depends on leaving original text visible and unrearranged so readers can notice, judge, and recover from automated saliency choices.Automated errors therefore appear as visible attribute choices rather than hidden omissions or altered text.Source: §6 · p. 24
  • Generalizing AI-resilient interfaces to additional tasks and domains remains an important direction for future work.The authors frame broader task and domain applicability as not yet established.Source: §6 · p. 24

A POST-STUDY SURVEY

The post-study survey includes questions about participants’ reading experience, with the final three rated on a 7-point agreement scale.

A POST-STUDY SURVEY

  • The final three survey questions measure agreement about participants’ reading experience on a 7-point Likert scale.Responses range from “Strongly Disagree” (1) to “Strongly Agree” (7).Source: Table 4 · p. 28

C MORE EXAMPLES OF GP-TSM

The paper provides additional GP-TSM renderings and a side-by-side comparison with NGP-TSM to illustrate the technique across GRE passages.

C MORE EXAMPLES OF GP-TSM

  • Figure 8 presents GP-TSM beside NGP-TSM as shown to a participant in a comparison interview.Source: Figure 8 · p. 29
  • Figure 9 shows GRE Passage 1 rendered using GP-TSM as an additional example of the technique.Source: Figure 9 · p. 30
  • Figure 10 shows GRE Passage 2 rendered using GP-TSM as an additional example of the technique.Source: Figure 10 · p. 31
  • Figure 11 shows GRE Passage 3 rendered using GP-TSM as an additional example of the technique.Source: Figure 11 · p. 32
Loading 2401.10873v1…