PaperlayerView exact revision on arXiv

Source-linked AI summary

Threddy: An Interactive System for Personalized Thread-based Exploration and Organization of Scientific Literature

Hyeonsu B. Kang, Joseph Chee Chang, Yongsung Kim, Aniket Kittur

arXiv:2208.03455v2cs.HC

TL;DR

Growing, branching literature makes research threads difficult to find, track, and organize without repeated context switches. Threddy captures citation context and references during reading, helping scientists curate threads and discover relevant papers while maintaining reading flow.

  • Problem

    As literature grows, finding and organizing multiple evolving research threads requires repeated reference tracking and can disrupt reading context.

    Source: §1 · pp. 1–2

  • Method

    Threddy lets readers collect authors’ pre-digested syntheses with associated references in situ and organize them into personal threads.

    Source: §1 · p. 2

  • Results

    Threddy increased users’ effectiveness in leveraging pre-digested syntheses, reduced context-switching costs, and heightened flow during literature review.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Threddy supports collecting and organizing citation-context clips and references while readers follow threads and discover additional papers.

    Source: §8 · p. 12

    Participants identified merged citation contexts as a challenge because one text patch can combine references with different levels of relevance.

    Source: §6.3 · p. 10

Abstract

from arXiv · show

Reviewing the literature to understand relevant threads of past work is a critical part of research and vehicle for learning. However, as the scientific literature grows the challenges for users to find and make sense of the many different threads of research grow as well. Previous work has helped scholars to find and group papers with citation information or textual similarity using standalone tools or overview visualizations. Instead, in this work we explore a tool integrated into users' reading process that helps them with leveraging authors' existing summarization of threads, typically in introduction or related work sections, in order to situate their own work's contributions. To explore this we developed a prototype that supports efficient extraction and organization of threads along with supporting evidence as scientists read research articles. The system then recommends further relevant articles based on user-created threads. We evaluate the system in a lab study and find that it helps scientists to follow and curate research threads without breaking out of their flow of reading, collect relevant papers and clips, and discover interesting new articles to further grow threads.

1 INTRODUCTION

Researchers need to track and make sense of growing, branching literature threads, but assembling and organizing them can interrupt reading. Threddy captures authors’ pre-digested syntheses and associated references in situ, helping scientists curate threads and discover further papers.

1 INTRODUCTION

  • Growing literature makes it challenging to find and track work across multiple, branching research threads, often requiring citation traversal, reading, and keyword searches.The work may span several related threads and sub-threads.Source: §1 · p. 1
  • Building threads from others’ summaries can require repeated reference lookups, paper searches, and note-taking, with irrelevant results forcing readers to rebuild their reading context.Citation notations may provide little context for identifying papers.Source: §1 · p. 2
  • As threads accumulate, readers must reconcile notes and references, track what is new, and find additional papers to grow each thread.Tabs do not preserve the citation context that explains the threads.Source: §1 · p. 2
  • Prior literature tools support search and paper collection, but the described systems do not use threads assembled during reading to recommend papers that grow those threads.The passage contrasts systems including Papers101, Apolo, PaperQuest, and VisualBib.Source: §1 · p. 2
  • Threddy integrates collection of citation context and extracted references into reading, letting users assemble personal research threads from other authors’ syntheses.The system is evaluated in a controlled lab study with 9 scientists conducting literature review in personalized domains.Source: §1 · p. 2
  • Threddy increased effectiveness in using pre-digested syntheses, reduced context-switching costs, and heightened users’ flow during literature review.These are the reported outcomes of the controlled lab study.Source: §1 · p. 3

2 RELATED WORK

Related work addresses literature discovery, interaction with documents, and reading support through tools ranging from document visualizations to standalone review environments. Threddy builds on these areas by integrating thread curation into reading.

2 RELATED WORK

  • Scholars reviewing literature face its immense scale and dynamic changes [7], which could overload them and make effective attention allocation imperative.Source: §2 · p. 3

2.1 Tools that Support Interaction within and across Documents

  • Document tools support reuse and visualization of selected content, narrative structure, and document collections, including Passages, Wang et al., and Apolo.Passages was favored over a baseline and encouraged reuse of collected text and structure.Source: §2.1 · p. 3

2.2 Tools that Augment Reading Experiences

  • Reading-augmentation systems reduce navigation and comprehension costs, while prior work points to reusing authors’ synthesis across papers without leaving reading flow.Examples include cross-referencing and table navigation [3], and margin discussions in NB.Source: §2.2 · p. 3

2.3 Tools for the Literature Review Workflow

  • Literature-review systems commonly provide standalone search, appraisal, and management environments, while ForSense and Fuse [38] add in-page sidebars for organizing clips and notes.PaperQuest, LitSonar, LitSense, and CiteSense support querying or managing review tasks.Source: §2.3 · p. 3

3 USAGE SCENARIO AND SYSTEM DESIGN

Threddy keeps thread-building support alongside paper reading, allowing users to capture context and references, reorganize threads, and discover related papers. Its design aims to reduce context switching while preserving the reader’s evolving organization.

3 USAGE SCENARIO AND SYSTEM DESIGN

  • In the scenario, recommendations lead the reader to add recent papers and revise her thread hierarchy as her understanding changes.The reader keeps the current thread visible while opening and reading another paper.Source: §3 · p. 4
  • Threddy’s PDF reader pairs a paper view with a persistent sidebar for thread-related content, so readers can move between papers while retaining their research context.The design rationale is to surface task context as readers move between papers.Source: §3 · pp. 4–5
  • Readers highlight text to extract its citation context and references, review or remove extraction errors, then add the context and papers as a new thread or to an existing one.They can also add only a clip or only the extracted papers to an existing thread.Source: Figure 2 · p. 4, §3 · p. 5
  • Readers can nest, move, relabel, and collapse threads; newly opened papers enter an “Unorganized Papers” thread for later organization.Recently changed threads move to the top of the thread drawer.Source: §3 · p. 5
  • The overview presents nested threads with their clips and references grouped by citation context, while recommendations appear alongside the collected material.The figure describes clips at the top, references in the middle, and recommendations below.Source: §3 · p. 5

4 SYSTEM ARCHITECTURE

Threddy combines PDF parsing and reference linking with in-reader thread organization and recommendations. Its architecture supports collecting citation context during reading and exploring additional papers from organized threads.

4.1 Automatic Extraction and Linking References to User Highlights

  • GROBID parsing identifies document structure, text, citations, figures, and tables; Threddy aligns parsed sentence coordinates with highlights to collect context and references.It uses S2ORC [40] and Semantic Scholar APIs to link references to paper metadata; previously processed PDFs are cached.Source: §4.1 · p. 5

4.3 Overview and Discovery of Additional Papers Related to Threads

  • The overview displays clips, references grouped by citation context, and nested threads, with recommendations generated from the papers in a selected thread.Users can add a recommendation and refresh results using the updated thread references.Source: Figure 3 · p. 6, §4.3 · p. 7

4.2 User Highlights and Creation of Threads

  • The sidebar’s holding tank lets users inspect extracted references, choose which to keep, and add the context or papers to a new or existing thread.Thread suggestions rank relevant top-level threads and sub-threads based on the highlighted text.Source: §4.2 · p. 6

4 SYSTEM ARCHITECTURE

  • The architecture connects PDF rendering, highlight handling, parsing, paper lookup, and recommendation, with data persisted as readers move between papers.The front end is a Chrome extension and the back end uses Flask and GROBID.Source: §4 · p. 5, Figure 5 · p. 7

4.2 User Highlights and Creation of Threads

  • Threads support nesting, moving, editing, and collapsing; papers opened in the viewer are initially placed in “Unorganized Papers” for deferred organization.Clips are hidden by default in the thread view to reduce clutter, while reference cards remain directly accessible.Source: §4.2 · pp. 6–7

4.3 Overview and Discovery of Additional Papers Related to Threads

  • Recommendations prioritize citation coverage, sample 50 top-ranked results, sort by recency, and use SPECTER similarity to break publication-year ties.The search is limited to 1,000 direct citations per thread reference; longer citation chains are left for future exploration.Source: §4.3 · pp. 7–8

5 EVALUATION

The evaluation compared Threddy with Google Docs in a within-subject literature-review task using topics participants personally wanted to explore. Researchers assessed outline creation, workflow experience, and system acceptance.

5 EVALUATION

  • A within-subject study compared Threddy with Google Docs, chosen as a familiar and accessible baseline for creating research threads, clipping text, and adding references.The task was to produce an outline for future use or for someone else to build upon.Source: §5 · p. 8
  • Nine active researchers completed two 20-minute tasks using personally motivating seed papers, with condition order counterbalanced across four Latin Square blocks.Participants were 1 Master’s student, 2 post-docs, and 6 PhD students across HCI, NLP, and Material Sciences.Source: §5 · p. 8
  • Training introduced threads, clips, and references, then asked participants to recreate a Google Docs outline in Threddy after a system tour.Threads could form hierarchies; clips could range from phrases to images, and references were papers relevant to a thread.Source: §5 · p. 8
  • Surveys measured task demand, technology acceptance, and interaction flow, alongside questions about extraction and organization of clips and references.Two authors coded baseline outlines for thread, clip, and reference counts; ordinal Krippendorff’s alphas were 0.842, 0.962, and 0.822, respectively.Source: §5 · p. 8

6 FINDINGS

Compared with Google Docs, Threddy increased the collection and organization of clips and references, while thread counts were similar. Participants also reported workflow and discovery benefits, alongside challenges when citation contexts combine distinct contributions.

6.1.1 Quantitative Results.

  • Clips averaged 9.9 vs 4.9 and references 20.4 vs 7.9 in Threddy vs Google Docs; thread counts did not differ significantly.The figure reports the comparison across threads, clips, and references.Source: §6.1.1 · pp. 8–9, Figure 6 · p. 9

6.1.2 Qualitative Results.

  • Automatic reference extraction, metadata linking, and formatting reduced cross-checking and application switching that participants described in their usual review workflows.Participants said saved time could be used to read additional papers or explore another thread.Source: §6.1.2 · p. 9
  • Persistent threads and clips helped participants retain context across papers and continue organizing their thinking while reading.Participants described the interface as a continuous stream of cognition that reminded them why a paper was relevant.Source: §6.1.2 · p. 9
  • Participants rated clip collection, reference tracking, reference organization, and flow higher with Threddy; overall task demand did not differ significantly.The reported flow composite means were 51.0 for Threddy and 42.9 for Google Docs.Source: §6.1.2 · pp. 9–10

6.2 Discovering Papers Relevant to Threads

  • Participants found recommendations accessible and often relevant, and rated finding additional papers for each thread easier with Threddy than Google Docs.Some participants valued newer papers citing thread references for gauging field progress, while one reported fear of missing out.Source: §6.2 · p. 10

6.3 Extracting Pre-digested Threads, Integrating Papers into Assembled Threads

  • Merged citation contexts can combine general and specific contributions, so fuzzy extraction may require users to remove unwanted references and disentangle batches.Participants also noted that metadata or TL;DRs could be unavailable or insufficient to understand the citation context.Source: §6.3 · p. 10

7 DISCUSSION

Threddy shifted participants toward following research threads across papers and supported collecting, organizing, and growing them during reading. Participants also identified limits around recommendations and long-term organization, while the authors discuss possibilities beyond citation chaining.

7.1 Impact on the Workflow

  • Participants used Threddy to explore and evolve research threads across papers without significant added workload, with support for collecting clips and relevant papers.The study found no significant condition difference in thread counts or perceived demand; participants reported significant support for auxiliary collection and organization tasks.Source: §7.1 · p. 10
  • Persistent threads and in-thread recommendations supported collecting and structuring information during reading, with reduced context switching and working-memory demands.The authors position this support as particularly useful during early literature-review foraging; concept mapping and synthesis tools may serve other stages or goals.Source: §7.1 · pp. 10–11
  • Thread-specific recommendations helped participants explore relevant literature and check whether their threads made sense, though some wanted greater control over recommendation context.Participants also used recommendations to consider redefining or subdividing threads; some found citation-coverage recommendations too broad.Source: §7.1 · p. 11

7.2 Scaling over a Long Period of Time

  • Participants refactored threads as their understanding changed, but reported difficulty viewing all threads and reorganizing them in a dedicated workspace.Participants discussed focusing on roughly 2–4 topic threads for a paper or grant; they split dense topics, renamed threads, and nested or moved them.Source: §7.2 · p. 11

7.3 Beyond Citation Chaining

  • The authors suggest that in-situ reading support may help scholars engage with relevant literature beyond familiar domains, while noting citation chaining can limit discoverability and risk filter bubbles.They discuss computational analogies and retrieval controls that could increase cross-domain recommendations.Source: §7.3 · pp. 11–12

8 CONCLUSION

Threddy supports collecting authors’ pre-digested synthesis and assembling personally relevant research threads from it. It integrates this work into reading, with further research needed to identify additional design implications.

8 CONCLUSION

  • Threddy collects text patches containing authors’ pre-digested synthesis, citation context, and automatically extracted associated references.These clippings help users assemble threads of personal interest from introductions or related-work sections.Source: §8 · p. 12
  • Unlike separate information environments, Threddy integrates thread collection and assembly into users’ in-situ reading context.Source: §8 · p. 12
  • Further research is needed to uncover additional design implications for in-situ support of collecting others’ synthesis and assembling it into users’ own threads.Source: §8 · p. 12

Appendix A. Algorithm for Ranking Relevant Existing Threads for Adding New Threads

The ranking algorithm identifies related vertical chains for a new thread, then orders individual threads within a selected chain by similarity. Its chain-level fit balances similarity to the centroid with similarity to the closest member.

Appendix A. Algorithm for Ranking Relevant Existing Threads for Adding New Threads

  • Figure 7 depicts two ranking stages: chain-level fit followed by member-level similarity ranking, with unmatched threads available for top-level insertion.Source: Figure 7 · p. 14
  • The algorithm ranks candidate chains using a fit that balances similarity to the chain centroid with similarity to its closest member.Threads are embedded with a MiniLM model, and the centroid is the average of member embeddings.Source: Appendix A · p. 14
  • After selecting a chain, the system ranks its member threads by similarity to the target and presents the ordering for insertion.Source: Appendix A · p. 14

Appendix B. Additional Survey Results

Appendix B describes additional questionnaire items and participant responses by condition, and notes that paired-sample tests were used for comparisons.

Appendix B. Additional Survey Results

  • Additional questionnaire items and participant responses are grouped by condition in Table 1.Source: Appendix B · p. 14
  • Two-sided paired-sample t-tests were used to calculate p-values between conditions.Source: Appendix B · p. 14
  • The appendix directs readers to Section 7.1 for discussion of the questionnaire results.Source: Appendix B · p. 14

Appendix C. Vignettes of Participants’ Threads

The vignettes show participants’ research threads across healthcare AI, model interpretability, and review research; they are simplified snapshots of structures that can change through iteration.

Appendix C. Vignettes of Participants’ Threads

  • The vignettes omit clips, references, and loose papers, and their thread structures may change as participants iterate.The appendix presents simplified snapshots rather than complete or fixed thread collections.Source: Appendix C · p. 14
  • Participant A’s threads span healthcare AI adoption barriers, AI onboarding, clinical decision support, and explainable AI.Other listed threads concern mental models for decision making and errors, and machine learning as a second set of eyes.Source: Appendix C · p. 14
  • Participant B’s threads range from interpretable model classes and generalized additive models to explainability methods and empirical studies.The vignette also names Explainable Boosting Machine and model interpretability broadly.Source: Appendix C · p. 14
  • Participant C’s threads cover table-based decision support, review summarization, aspect extraction, and consumer-product review research.The vignette also includes sensemaking of collections of online information.Source: Appendix C · p. 14
  • Questionnaire responses suggest Threddy helps collect clips, find references to grow threads, and organize them, but not reduce task demand.The table caption also reports no significant difference in compatibility or adoption likelihood between Threddy and Google Docs.Source: Table 1 · p. 15

Related papers

Loading 2208.03455v2…