Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents

Motivation

• Some expect GenAI to help BLV screen-reader users overcome inaccessible or poorly contextualized documents, but question-answer tools may skew access toward selected content and leave collections partially understood.The introduction argues that understanding how GenAI shapes BLV users’ information exploration and consumption is important for designing accessible technologies [42].Source: §1 · p. 1

Research questions and contributions

• The study examines how interface type affects screen-reader users’ browsing, knowledge integration and application, and interface preferences, contributing empirical evidence about accessible information access.Its overarching goal is to compare traditional document browsing with LLM-driven conversational access.Source: §1 · p. 2

Study design

• A mixed-methods study with 16 screen-reader users compared DI and QAI across browsing logs, concept maps, decision scenarios, and interviews in the fictional worlds Solana and Dominion.The four phases examined browsing patterns, mental-model construction, knowledge application, and participants’ overall experiences and preferences.Source: §1 · p. 2

Findings

• Participants explored a wider range of documents and created larger, broader, more accurate concept maps with DI, while QAI yielded denser links and broader transitions within fewer documents.Most participants did not detect these differences; subjective perceptions were divided, with some preferring DI’s explicit structure and others QAI’s conversational style.Source: §1 · p. 2
• Most participants did not detect the interface differences, and preferences split between DI’s explicit structure and QAI’s conversational style.Subjective perceptions of the interfaces were divided, reflecting differences in exploration behaviors, strategies, and experiences.Source: §1 · p. 2

2.1 Mental Models and Mental Model Elicitation

• Mental models organize knowledge to guide action, and concept mapping externalizes concepts and links to reveal internal representations beyond retrieval-only verbal protocols.Prior work on blind users largely examined interaction through verbal protocols, observations, and performance data rather than externalized representations; concept mapping addresses the limits of verbal protocols alone.Source: §2.1 · p. 2

2.2 Accessing Documents and Information Access Technologies

• Browsing supports exploratory learning without a specific query, while GenAI can synthesize across documents conversationally; how LLM mediation shapes accessible exploration of linked corpora remains unclear.Prompt-and-summary interaction may narrow document exposure and weaken exploration, comparison, and continuity [61]; the study therefore compares GenAI question answering with document-based interaction.Source: §2.2 · pp. 2–3

2.3 Document Accessibility

• Accessible document design uses screen-reader annotations and tactile graphics, whereas LLM descriptions and summaries ease navigation but may expose only a subset of content.The study’s document interface included sufficient annotations and a tactile overlay; earlier voice assistants supported quick, simple tasks [84], while LLMs may produce fragmented understanding [61].Source: §2.3 · p. 3

3 Study Methodology

• Across two laboratory sessions, participants used both interfaces in a counterbalanced within-subjects study of navigation, mental-model building, and knowledge application.The researchers used the controlled setting to gather detailed data and compare the access paradigms.Source: §3 · p. 3, Figure 1 · p. 4

3.1 Study Materials: Fictional Worlds

• Researchers created the Solana and Dominion information spaces from scratch to limit prior familiarity while providing diverse content and depth for extended exploration.The sets comprised interconnected documents modeled on hyperlinked pages and fictional worlds.Source: §3.1 · p. 3
• Each world contained 25 documents, including an index, with 8 Solana and 6 Dominion diagrams alongside text on topics such as history, governance, and culture.The text resembled Wikipedia articles, with headings and inter-document links.Source: §3.1 · p. 3

3.2 Interfaces (Main Condition)

• The interfaces were designed to isolate direct corpus exploration from LLM-mediated question answering, and to represent the best reasonable alternatives available or soon to be available.Participants used their own devices and screen readers to access WCAG 2.1-compliant web pages, reducing training needs and supporting ecological validity.Source: §3.2 · p. 4, §3.2.1 · p. 4

3.2.1 Document Interface (DI).

• The DI paired screen-reader-accessible text pages with tactile spatial diagrams, linking the two media through browser navigation and verbal switch prompts.Finger movements over the 3D-printed overlays produced verbal descriptions of labeled areas, while preserving spatial relationships difficult to communicate through screen-reader navigation alone.Source: §3.2.1 · p. 4

3.2.2 Question-Answer Interface (QAI).

• The QAI let participants type or speak questions and returned source-based responses as text, bullets, tables, image descriptions, or direct quotations.When it could not answer, it suggested related documents or stated that the topic was unavailable and offered alternatives.Source: §3.2.2 · pp. 4–5
• The QAI used Gemini 2.5 Pro with cache-augmented generation over the 25-document corpus, and researchers observed no hallucinations or erroneous responses.A custom prompt constrained answers to the corpus; the system loaded spatial documents into context and used related-document suggestions when direct answers were unavailable.Source: §3.2.2 · pp. 4–5

3.3 Participants

• 16 participants aged ≥19 reported regular use of at least one screen reader, including JAWS, NVDA, VoiceOver, or TalkBack.Source: §3.3 · p. 5
• Participants reported varied GenAI uses: accessibility features (n=11), work or productivity (n=8), technical assistance (n=7), writing or content generation (n=5), and information seeking (n=4).Source: §3.3 · p. 5
• Recruitment used community organizations, social media, and snowball sampling; participants received $40 per session, and the study had local ethics-board approval.Source: §3.3 · p. 5

Procedure

• Each participant completed two 90-minute sessions on separate days, with interface and world order counterbalanced and sessions held one to three days apart.Sessions were capped at five days apart to reduce cognitive load, fatigue, and carryover while allowing flexible scheduling and participant choice of location, device, and screen reader.Source: §3.4 · p. 5

Phase 1

• In Phase 1, participants freely explored the assigned world and prioritized acquiring domain knowledge over recall or constrained task performance.QAI participants were told its underlying content comprised 25 documents; participants could take notes, but none did.Source: §3.4 · p. 5

Phase 2

• In Phase 2, participants described the world and its important concepts and relationships without interface access, while the researcher built and validated a participant-led concept map.A structured script limited the researcher to concepts and relationships introduced by the participant, reducing bias and eliciting mental-model organization beyond recall.Source: §3.4 · pp. 5–6

Phase 3

• In Phase 3, participants answered a world-specific decision scenario to assess how effectively they could apply their acquired knowledge.Source: §3.4 · p. 6

Phase 4

• In Phase 4, participants discussed their interactions, ease of use, challenges, and overall experience; after session two, they also compared the interfaces and reported preference, perceived learning support, and confidence.Source: §3.4 · p. 6

3.5.1 Document Traversal Data.

• Traversal analysis measured document visits and transitions, plus graph structure, to characterize how participants moved through each information space.Measures included unique and total document visits and transitions, transitions per document, density, directed diameter, degree of variance, and average shortest path length. In DI, opening a document counted as a visit; in QAI, a response counted when it contained information corresponding to at least five contiguous source-document sentences. Transitions were link traversals in DI and changes in mapped source documents in QAI; self-loops were excluded.Source: §3.5.1 · pp. 6–7

3.5.2 Concept Maps.

• Concept maps measured participants’ stated topics, subtopics, connections, and errors to assess the structure and accuracy of their mental models.Topics and subtopics counted when unambiguously mapped to predefined documents; repeated mentions did not count again, while beyond-document claims were marked inaccurate. Researchers verified additional connections through probing; errors included contradictions, unsupported inferences, and incorrect detail recall, with repeats counted only when they added a distinct claim or connection.Source: §3.5.2 · p. 7

3.5.3 Decision-based Tasks.

• Decision-task responses were assessed using concept-map counts and error metrics, but not graph measures because responses were sparse and some participants could not respond.The authors deferred discussion of why participants could not produce responses to the Results section.Source: §3.5.3 · p. 7

3.5.4 Qualitative Responses.

• Qualitative analysis used interaction records and graph visualizations for RQ1, and thematic interview coding for RQ2 and RQ3.Thematic coding combined inductive and deductive approaches, iteratively refined a codebook, and applied the final 93-code codebook to remaining transcripts while revisiting earlier ones for consistency. Participants also compared the interfaces on preference, exploration, learning, and confidence in answers, then explained their choices.Source: §3.5.4 · p. 7

3.6 Positionality Statement

• The authors grounded the study in BLV screen-reader users’ own accounts, practices, and perspectives, while none of the authors identified as BLV.The second author brought more than six years of engagement with BLV communities, and the fourth author brought assistive-technology design experience.Source: §3.6 · p. 7
• Authors’ expertise shaped study design, accessible materials, qualitative coding, knowledge-elicitation tasks, and statistical analysis.The second author’s qualitative background guided coding; the third and fourth authors’ expertise informed statistical analysis.Source: §3.6 · p. 7
• The authors treat technology as an active mediator: it can reduce barriers, but restrictive design can create disability and impose dysaffordances.Source: §3.6 · p. 7

• Each results subsection presents quantitative measurements first, followed by qualitative analysis.Source: §4 · p. 8

4.1.1 Quantitative Results.

• Participants visited 11.44 unique documents with DI versus 9.31 with QAI, a 22.9% increase, despite similar total visits.This supports broader exploration with DI; the difference in unique documents was significant (t(15) = 2.34, p = 0.03, η2 = 0.27).Source: §4.1.1 · p. 8
• QAI navigation graphs had higher density (0.21 vs. 0.13), lower directed diameter (4.94 vs. 7.75), and shorter average shortest path length (2.44 vs. 3.77) than DI graphs.Transition counts were not conclusively different (all p > 0.05), despite QAI answers being able to draw on multiple documents at once.Source: §4.1.1 · p. 8

4.1.2 Qualitative Results.

• For overview, DI helped users accept the document structure (n = 9), while QAI made self-defined overviews easier (n = 7) and DI could feel frustrating (n = 5).Some QAI users did not know what to ask for an overview (n = 9) or realize that requesting one could help (n = 6).Source: §4.1.2 · p. 8
• For depth, DI supported following a predetermined structure (n = 7), whereas QAI made self-defined paths easier (n = 7) than manually tracing documents in DI (n = 9).With QAI, users could keep asking focused questions, but found it difficult to know when a topic had been exhausted (n = 8).Source: §4.1.2 · p. 9
• For fact finding, QAI readily provided contextualized answers (n = 9), while DI required users to search across documents (n = 7), regardless of their preference for document structure.The three recurring exploration patterns were overview, depth, and fact finding; overall, DI offered accessible pre-established structure, while QAI facilitated custom queries across scattered information.Source: §4.1.2 · pp. 8–9, §4.1.3 · p. 9

4.2.1 Concept Maps (Phase 2).

• DI concept maps contained 67% more topics and 11% more subtopics, and 38% more total connections than QAI maps, but similar connections per topic.Participants averaged 9.44 topics with DI versus 5.63 with QAI, and 20.69 connections versus 14.94; connections per topic were 1.09 versus 1.03.Source: §4.2.1 · pp. 9–10, Figure 9 · p. 10
• DI maps had 54% lower error rates (0.12 vs 0.26) and fewer raw errors (3.31 vs 6.00) than QAI maps; after errors were removed, they were less interconnected.DI graphs had lower density, higher diameter and average shortest path length, and higher degree variance (all p<0.04).Source: §4.2.1 · p. 10, Figure 10 · p. 10

4.2.2 Decision-based Scenario (Phase 3).

• In the harder scenario task, six participants could not give coherent responses in one or both sessions, and DI’s higher-connection pattern was not statistically distinguishable.The researchers did not calculate graph metrics because the resulting graphs were sparse and noisy.Source: §4.2.2 · p. 10

4.2.3 Qualitative Results.

• DI’s visible structure helped participants understand the information space but could frustrate those wanting to build their own structure; QAI enabled self-directed structuring but required skill in what and how to ask.Five participants appreciated DI’s visible organization, while four found it frustrating; seven valued QAI’s question-led structure, while ten noted the questioning demands.Source: §4.2.3 · p. 10
• Participants said DI’s greater information volume helped them connect material, though links could leave relationships implicit; QAI directly integrated topics but required skillful questions.Some QAI users said simplistic questions led them in circles, while DI users noted that following links did not necessarily reveal how topics related.Source: §4.2.3 · pp. 10–11
• Some participants said DI’s stable structure helped them answer scenarios, while QAI users valued asking personally important questions but often doubted whether their answers were sufficient or correct.Participants also said successful application with DI required actively processing the information rather than simply absorbing it.Source: §4.2.3 · p. 11

4.2.4 RQ2 Synthesis.

• Overall, DI supported stronger knowledge application, while QAI offered flexible, smoothly connected answers but left participants uncertain whether their information gathering was exhaustive.The synthesis attributes DI’s advantage to the effort of connecting information as readers go, while QAI’s tailored responses depend on asking effective questions.Source: §4.2.4 · p. 11

4.3 What Aspects of the Different Interfaces Affect Preferences? (RQ3)

• Eight participants preferred each interface, and stated preference had no obvious mapping to perceived advantages in exploration, mental-model building, or confidence; perceptions also diverged from measured patterns.The mismatch appeared for breadth of exploration and more strongly for perceived learning support and confidence in answering questions.Source: §4.3 · p. 11
• Participants framed preferences around agency/control and effort/cognitive load, using both themes to support either interface.Source: §4.3 · p. 11
• Nine participants valued self-direction in both interfaces—choosing documents and links in DI or asking freely in QAI—while DI freedom could misdirect (3) and QAI steering could be difficult (6).Source: §4.3 · pp. 11–12
• Participants valued DI’s straightforward access (5) but cited browsing and tracking burden (4), while QAI could address several topics at once (9) but required effective probing (5).Source: §4.3 · p. 12

5.1 Answers to RQ’s and their connections

• DI users visited more unique documents, while QAI enabled flexible transitions; QAI yielded smaller, more erroneous mental models and poorer application, despite tighter connections.Participants may have performed differently if task instructions had explicitly emphasized memorization, which warrants further study.Source: §5.1 · p. 12
• Participants often rated QAI as better for outcomes where they did worse, while preferences were evenly split between the interfaces.The authors suggest interface value may be obscured from users, or preferences may reflect factors beyond mental-model completeness and application.Source: §5.1 · p. 12

5.2 What does this mean for GenAI interfaces and non-visual information access?

• QAI supported contextualized fact-finding across documents, whereas DI better supported broad corpus exploration; QAI may hinder open-ended learning and expertise-building, especially for BLV users.The authors note that GenAI adoption and the tedium of non-visual document access may increase the likelihood that BLV users replace document access with GenAI; differences from sighted users need research.Source: §5.2 · p. 12
• DI interaction helps users infer document architecture and coverage, while QAI focuses exploration on prompts and early choices, making completeness harder to assess.Although participants experienced QAI as efficient, flexible, and responsive, these benefits did not translate into larger concept maps or stronger application performance, and users often judged interface support poorly.Source: §5.2 · p. 13

5.3 Implications for users and future directions for practitioners

• Users should be warned that QAI may encourage superficial processing and reduce ability to apply information, while recognizing that not all readers seek better models or memory.The authors emphasize this risk for BLV users seeking technology to avoid being perceived as less effective than sighted peers.Source: §5.3 · p. 13
• Future systems could scaffold learning, adapt agency and effort to user preferences, or combine document navigation, search, and conversation for different tasks.The authors also propose studying whether document structures should change as QAI becomes a dominant access mode.Source: §5.3 · p. 13

5.4 Limitations

• Interpret DI advantages cautiously: touch-enabled spatial documents, corpus overviews, visit-counting choices, no BLV community review, SR-only recruitment, and two 90-minute sessions limit attribution and generalizability.The authors call for further experiments, including ecologically faithful evaluations of systems combining DI and QAI features.Source: §5.4 · p. 13

6 Conclusion

• QAI conversation felt deceptively expansive without improving learning, while DI’s structure supported more comprehensive mental models despite potentially greater effort or overwhelm.The contrast matters because conversational coverage is synthesized, whereas DI provides structure for building a more comprehensive model.Source: §6 · p. 13
• Many participants perceived QAI as better despite performance suggesting otherwise, creating a risk that interfaces that feel easier may support weaker understanding.This risk is especially relevant for BLV users seeking to reduce the burden of traditional document access.Source: §6 · p. 14
• Accessible technologies should be evaluated for accurate understanding and meaningful exploration, not only speed or convenience, as LLM interfaces become intermediaries.Source: §6 · p. 14