PaperlayerView exact revision on arXiv

Source-linked AI summary

CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Context

Joseph Chee Chang, Amy X. Zhang, Jonathan Bragg, Andrew Head, Kyle Lo, Doug Downey, Daniel S. Weld

arXiv:2302.07302v1cs.HCcs.DL

TL;DR

Literature reviews can involve hundreds of citations, making it difficult to prioritize which ones fit a reader’s current interests. CiteSee personalizes citation highlights and historical context from users’ publication, reading, and saving activities. A lab study found it more effective than three baselines for paper discovery, and a field deployment reported better situational awareness during exploration.

  • Problem

    During literature reviews, readers encounter many citations but lack in-situ support for deciding which are relevant to their evolving interests.

    Source: §1 · pp. 1–2

  • Method

    CiteSee uses users’ publication, reading, and paper-library histories to augment inline citations and provide persistent personalized context.

    Source: §1 · p. 2, §4 · p. 5

  • Results

    CiteSee’s reading-history citation strategy was significantly more effective for discovery than three baseline strategies, and field participants valued its context for better situational awareness.

    Source: §8 · p. 14

  • Takeaways & Limitations

    Personalized citation cues and context helped field participants keep track of explorations while conducting real-world literature reviews.

    Source: §8 · p. 14

    CiteSee’s history window does not directly support interleaved reading across multiple tasks; future work could let users specify review contexts or identify relevant history automatically.

    Source: §7 · p. 13

Abstract

from arXiv · show

When reading a scholarly article, inline citations help researchers contextualize the current article and discover relevant prior work. However, it can be challenging to prioritize and make sense of the hundreds of citations encountered during literature reviews. This paper introduces CiteSee, a paper reading tool that leverages a user's publishing, reading, and saving activities to provide personalized visual augmentations and context around citations. First, CiteSee connects the current paper to familiar contexts by surfacing known citations a user had cited or opened. Second, CiteSee helps users prioritize their exploration by highlighting relevant but unknown citations based on saving and reading history. We conducted a lab study that suggests CiteSee is significantly more effective for paper discovery than three baselines. A field deployment study shows CiteSee helps participants keep track of their explorations and leads to better situational awareness and increased paper discovery via inline citation when conducting real-world literature reviews.

1 INTRODUCTION

Inline citations support literature discovery, but readers face many citations and lack in-situ help judging which matter to their evolving interests. CiteSee personalizes citation cues and historical context; studies report stronger discovery than three baselines and promising field-deployment outcomes.

1 INTRODUCTION

  • Readers must prioritize among dozens or hundreds of citations, although only a subset may fit their interests at the time of reading.Existing tools do not track which citations readers have encountered or preserve the contexts in which they encountered them.Source: §1 · pp. 1–2
  • CiteSee uses a reader’s publication, library, and reading history to personalize inline citation cues and persistent context.It connects citations to previous activities, including where a paper was saved or how other read papers described it.Source: §1 · pp. 2–3
  • The paper targets the gap between non-personalized citation analysis and paper-discovery tools that operate independently of reading.Its goal is in-situ support for deciding which citations are worth consulting in light of readers’ interests and search history.Source: §1 · pp. 2–3
  • The lab study found citation highlighting significantly more effective for discovery than three baselines, while the field study reported a 2.7x increase in inline-citation discovery versus previously reported self-reported numbers.The field deployment also offered qualitative insights into how participants prioritized and tracked explorations.Source: §1 · p. 3

2 RELATED WORK

CiteSee combines ideas from reading interfaces, exploratory search, sensemaking, and recommender systems to support citation exploration during reading. Unlike prior tools focused on single-document context or standalone discovery, it personalizes outgoing citations using reading history.

2 RELATED WORK

  • CiteSee treats literature review as exploratory, with readers’ goals crystallizing over time as they skim and switch among papers.It draws on sensemaking and information-foraging models to guide citation prioritization.Source: §2 · p. 3

2.1 Scientific Paper Reading Interfaces

  • Unlike interfaces that support cross-referencing within one paper, CiteSee helps readers connect repeated outgoing citations across papers and see how they were described.CiteRead instead surfaces incoming citations from follow-on papers without personalization.Source: §2.1 · p. 3

2.2 Paper Recommendation and Exploration

  • Standalone recommenders and citation-graph tools support discovery outside reading, whereas CiteSee prioritizes inline citations in-situ using reading history and engagement.Its approach adapts ranking signals without requiring users to specify seed papers or interests explicitly.Source: §2.2 · pp. 3–4

3 PRELIMINARY INTERVIEWS

Five researchers were interviewed using paper-reader design probes to understand citation use during literature reviews. Their concerns about missed work and lost context informed three design goals for CiteSee.

3 PRELIMINARY INTERVIEWS

  • Five researchers with varied backgrounds completed think-aloud reading and design-probe interviews, which were analyzed thematically.The probes varied citation-highlighting strategies and Paper Card context.Source: §3 · pp. 4–5

3.1 Fear of Overlooking Important Citations

  • Participants feared overlooking important prior work, including work not highly cited globally but familiar within their research community.They cited the risk of duplicating explored approaches or missing work “everyone else” cites.Source: §3.1 · p. 5

3.2 Progress Tracking and Loss of Context

  • Participants struggled to remember why papers were saved and saw value in consistent annotations and persistent citing-sentence context across documents.Existing strategies included browser tabs, copied titles, and libraries or folders.Source: §3.2 · p. 5

3.3 DESIGN GOALS

  • The resulting goals were to surface relevant unknown citations, connect known citations to familiar contexts, and preserve personalized context across encounters.The proposed context includes how a cited paper appears across papers previously read.Source: §3.3 · p. 5

4 SYSTEM DESIGN

CiteSee links citations to a reader’s history and library, then augments inline citations and Paper Cards to support discovery, familiarity, and persistent context. Its mechanisms use adjustable reading-history signals and distinguish explored papers from unexplored ones.

4 SYSTEM DESIGN

  • CiteSee visually augments citations to previously explored and important unexplored papers, with Paper Cards showing interaction history and citing sentences.Users can click inline citations to access personalized context such as when a paper was last opened.Source: §4 · pp. 5–6

4.1 Overview of Citation Augmentation Types

  • The system marks reencountered citations with yellow-to-orange shades, visited papers green, saved papers red, user-cited papers with quotation marks, and the user’s own papers with hearts.When multiple cues apply, saved status takes precedence over visited status, while known papers suppress reencounter highlighting.Source: §4.1 · p. 6, Figure 4 · p. 6

4.2 Example User Scenario

  • In the example scenario, citations to opened papers appear green, saved papers red, and unexplored papers receive yellow shades based on how many recent readings cite them.A Paper Card’s citing sentences from recent papers help assess whether a potentially relevant dataset remains in use.Source: §4.2 · pp. 6–7

4.3 [D1] Discover Relevant Citations

  • CiteSee compares current-paper citations with citations in recently opened papers, highlighting repeated citations and allowing readers to adjust the history window from a default of 20 papers.Highlight intensity also reflects estimated reading progress and saved status; the score is capped at five points.Source: §4.3 · p. 7, Figure 5 · p. 7

4.4 [D2] Surfacing Familiar Papers

  • Visited and saved papers are marked green and red, while a reader’s publication history identifies their own papers and papers those publications cited.The cues update in real time when a paper is opened or saved.Source: §4.4 · p. 7

4.5 [D3] Paper Cards with Personalized Context

  • Paper Cards show other read papers that cite a reencountered work, their citing sentences, opening times, and estimated reading progress.When a paper is saved from a card, CiteSee retains the discovery paper and sentence as context for later encounters.Source: §4.5 · pp. 7–8

4.6 Implementation Details

  • CiteSee is a Chrome extension using Grobid to parse PDFs and Semantic Scholar APIs for paper metadata, libraries, and publication histories.Unknown-domain and local PDFs require user permission for upload; uploaded files are discarded after processing.Source: §4.6 · p. 8

5 STUDY 1: DISCOVER RELEVANT CITATIONS

Study 1 tested whether highlighting citations reencountered across papers in a reader’s history helps identify useful literature-review citations. The strategy outperformed three baselines, though the lab task was shortened to manage fatigue and time.

5 STUDY 1: DISCOVER RELEVANT CITATIONS

  • The lab task simulated literature review in two passes over three papers, rating five citations per strategy across four selection strategies.The strategies were reencountered citations, linear reading, global citation count, and Specter similarity.Source: §5 · pp. 8–9

5.1 Study 1 Limitations

  • Participants rated citations without being told how they were selected; Study 2 then deployed the winning strategy during participants’ real-world tasks.Afterward, participants reported that they had assumed a single selection method.Source: Figure 6 · p. 9, §5.1 · p. 10
  • To limit fatigue and fit the 60-minute study, the final design used two passes through the first parts of three papers rather than deep reading four papers.Only citations in the Introduction and Related Work sections were tested.Source: §5.1 · p. 9

5.2 Study 1 Results

  • Ten participants supplied 417 five-point ratings, modeled with cumulative-link mixed effects for LitReview and CurrPaper.The model included participant-specific random effects and separate models for the primary and secondary measures.Source: §5 · p. 9, §5.2 · p. 10
  • Reencountered significantly outperformed all three baselines on both LitReview and CurrPaper ratings.For LitReview, Reencountered had β=2.23 (p=3.3e−22); its coefficient exceeded Specter’s by 1.59 (p=3.91e-8).Source: Figure 8 · p. 10, §5.2 · p. 11

6 STUDY 2: FIELD DEPLOYMENT

In a one-to-two-week field deployment with six participants, CiteSee supported citation exploration, paper triage, and remembering saved work. Participants also described improved awareness of connections across papers and continued voluntary use after the study.

6 STUDY 2: FIELD DEPLOYMENT

  • Six participants used CiteSee on personal and work computers for one to two weeks, then completed retrospective interviews analyzed thematically.Participants were asked to keep diaries during deployment; the study aimed to understand real-world use and costs and benefits, not estimate theme prevalence.Source: §6 · p. 11

6.1 Study 2 Results

  • 57% (n=6, SD=24%, .95CI[32%,82%]) of saved papers came from inline citations; participants discovered useful prior work via inline citations around 2.7x more frequently than a prior survey.Although reencountered citations averaged 8.2% of inline citations, they accounted for 37% of Paper Cards accessed (4.5 times).Source: §6.1 · p. 11

6.1.1 Global Citation Counts vs Reencountered Citations (D1,D3).

  • Participants viewed global citation counts mainly as a paper-quality proxy and reencountered citations as a better signal of relevance; one participant increasingly relied on the latter as her history grew.P2 distinguished the question of whether a paper is worth reading from how it relates, which titles and citing sentences helped explain.Source: §6.1.1 · p. 12

6.1.2 Highlighting based on Engagement with other Papers (D1).

  • Different highlight shades reflected users’ perceived relevance from engagement with citing papers, but time spent on an ultimately irrelevant cross-field paper could create a mixed signal.Participants could remove papers from reading history, but some did not want to lose that history or thought removal was not worth the effort.Source: §6.1.2 · p. 12

6.1.3 Triaging between and within Papers (D1,D2).

  • The summary of augmentation counts helped participants triage papers, while citation highlights concentrated in relevant sections supported skimming.Participants said the highlights helped them decide which parts of a paper deserved attention.Source: §6.1.3 · pp. 12–13

6.1.4 Remembering Papers from the Past (D2,D3).

  • Visited and saved indicators helped participants track which papers they had explored, while saved citing sentences supplied context that titles and abstracts often lacked.Participants also suggested that attaching their own notes or selected sentences could further customize saved context.Source: §6.1.4 · p. 13

6.1.5 Sensemaking across Multiple Papers (D1,D2,D3).

  • Participants used CiteSee to find relevant prior work and gain awareness of citation-network connections, including central papers and subdomains.One participant described finding HCI-relevant papers through CiteSee rather than irrelevant results from a broad search query.Source: §6.1.5 · p. 13

6.1.6 Volunteered Continued Usage in the Wild.

  • Four of six participants continued using CiteSee voluntarily for more than two months after the study.At the time of writing, their continued use ranged from 62 to 121 days.Source: §6.1.6 · p. 13

7 LIMITATIONS AND FUTURE WORK

CiteSee’s current history-based approach has boundaries in supporting readers who alternate among tasks and in avoiding over-saturated highlights. Future work could improve relevance prediction, mitigate citation echo chambers, and extend support to note-taking and synthesis.

7 LIMITATIONS AND FUTURE WORK

  • Supporting readers who interleave multiple tasks remains a future direction; the current slider adjusts how much reading history CiteSee considers.The five-point reencounter heuristic can become over-saturated; opening or saving a reencountered citation removes its yellow highlight, and users can remove highlights from paper cards.Source: §7 · p. 13
  • Longer deployments could test more sophisticated reading-behavior models, but users may be uncertain when manually judging relevance, and better highlight accuracy may not provide significant benefit.Proposed relevance ratings could be compared with behavior traces such as reading time and mouse hovering.Source: §7 · p. 14
  • CiteSee’s citation-based recommendations can potentially create echo chambers, motivating semantic-similarity signals to surface work less represented in reading histories.The authors also identify recommendation placement and search-query suggestions as possible directions, while noting that suggestions must not disrupt reading flow.Source: §7 · p. 14
  • CiteSee focuses on literature exploration and discovery; extending it to support manual note-taking and cross-paper synthesis is a possible direction.The authors suggest concept cards that gather prior notes and relevant paragraphs for concepts mentioned in the current paper.Source: §7 · p. 14

8 CONCLUSION

CiteSee uses researchers’ prior activities to personalize inline citations, helping contextualize current papers and explore relevant prior work. A lab study found it significantly more effective for discovery than three baselines, while a week-long deployment showed better situational awareness during real-world literature reviews.

8 CONCLUSION

  • CiteSee personalizes inline citations using users’ prior research activities to help contextualize the current paper and explore relevant prior work.Its design combines ideas from non-personalized, single-document intelligent reading interfaces and paper recommender systems focused on non-reading discovery.Source: §8 · p. 14
  • CiteSee’s reading-history strategy was significantly more effective for discovering prior work than three baseline strategies.In a week-long field deployment, participants valued its personalized citation context, which allowed better situational awareness while exploring many papers.Source: §8 · p. 14

Related papers

Loading 2302.07302v1…