PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative AI Steering and Human Authoring: Complementary Search-as-Learning Scaffolds in Cross-Session Aggregated Academic Search

Milad Momeni, Orland Hoeber

arXiv:2610.12755v1cs.HC

TL;DR

Learners returning to aggregated academic search must reconstruct context from heterogeneous saved materials, and it remains unclear how consolidation scaffolds affect learning after resumption. The paper compares human-authored notes with steerable GenAI summaries in a two-session study. The approaches show contrasting outcome profiles rather than one overall best approach.

  • Problem

    The study addresses how allocating consolidation work between learners and GenAI shapes post-resumption learning as users return to heterogeneous saved search materials.

    Source: §1 · pp. 1–3

  • Method

    A controlled two-session study compared learner-authored notes with steerable GenAI summaries in an aggregated academic search interface.

    Source: §1 · p. 3, §3.3 · p. 8

  • Results

    STAIS showed better cross-session support and broader conceptual coverage, while NT was associated with richer inter-concept connections and higher self-reported certainty; neither was the overall best approach.

    Source: §5.5 · p. 16, §6 · p. 18, §7 · p. 20

  • Takeaways & Limitations

    Consolidation support should align with learner goals: GenAI scaffolding for rapid re-entry and conceptual mapping, and human-authored consolidation for relational integration and explanatory structuring.

    Source: §7 · p. 20

    The study bundled multiple scaffold features, so observed differences cannot be attributed narrowly to GenAI support, agency, or any single component.

    Source: §6.5 · p. 19

Abstract

from arXiv · show

Aggregated academic search interfaces combine resources from heterogeneous sources, such as scholarly libraries and reference encyclopedias, into a single workspace. Such tasks frequently span multiple sessions, requiring learners to return to and resume work after hours or days. End-of-session consolidation, the activity of organizing and synthesizing gathered information so it can be resumed later, is important in this setting but underexplored in search-as-learning. Generative AI (GenAI) has the potential to make consolidation more efficient, but it may also shift cognitive work away from the learner. This study compares two consolidation scaffolds embedded in the workspace framework of an existing academic search interface that aggregates and clusters saved search results from two different sources (our university library and Wikipedia), which was developed in earlier work. STAIS (Steerable AI Summaries) provides a workspace enhancement that produces GenAI summaries that users can correct, reorganize, and fact-check, automating parts of the consolidation activity. NT (Note-Taking) provides a workspace enhancement that includes rich-text note editors that learners can use to author their own consolidations of what they have found. In a controlled between-subjects study (n=36), participants completed two sessions one week apart while searching an unfamiliar topic using either STAIS or NT. The two conditions produced contrasting post-resumption outcome profiles: STAIS supported faster task resumption and was associated with broader conceptual coverage and higher self-reported domain knowledge, whereas NT was associated with richer inter-concept connections and higher self-reported certainty.

1 Introduction

The paper examines how learner-authored versus GenAI-steered consolidation shapes resumption and post-resumption learning in aggregated academic search. It compares these scaffolds in a two-session study and reports complementary outcome profiles.

1 Introduction

  • Returning to multi-session academic search requires learners to reconstruct prior findings, organization, priorities, and open questions from workspace cues.Aggregated search heightens this challenge because saved materials span heterogeneous sources and may be numerous.Source: §1 · pp. 1–2
  • Consolidation links cross-session resumption with search-as-learning because organizing encountered information can shape both reacquaintance and later learning.The paper defines it as end-of-session externalization and organization for later resumption, not offline memory consolidation.Source: §1 · p. 2
  • The study asks whether steerable GenAI consolidation changes post-resumption learning relative to human-authored notes, beyond potentially improving task resumption.The concern is whether delegating work weakens knowledge construction or whether learner inspection and correction redistribute rather than remove effort.Source: §1 · p. 2
  • A controlled study compared human-authored notes with steerable GenAI summaries in an aggregated academic search interface across two sessions one week apart.Participants searched an unfamiliar topic; the conditions shared the interface and differed in their consolidation scaffold.Source: §1 · p. 2
  • STAIS was associated with broader conceptual coverage and faster resumption, whereas NT was associated with richer inter-concept connections and higher self-reported certainty.The paper frames these as complementary profiles rather than evidence that one approach is outright better.Source: §1 · p. 3

2 Related Work

Prior work connects learning during search with the challenge of resuming suspended tasks, but leaves unclear how the source of consolidation cues affects learning after return. This study addresses that gap by varying the scaffold within a shared search environment.

2 Related Work

  • Prior work establishes workspace context and GenAI summaries as supports for resumption, but how learner-system allocations of consolidation work affect post-resumption learning remains unclear.The gap spans search-as-learning, cross-session search, and GenAI-assisted sensemaking.Source: §2 · p. 3

2.1 Search-as-Learning and the Anderson and Krathwohl (A&K) Taxonomy

  • The Anderson and Krathwohl taxonomy distinguishes knowledge types from cognitive processes, while search learning often involves understanding concepts and analyzing their relationships.Consolidation that connects and interprets ideas may support deeper conceptual learning than recording isolated facts.Source: §2.1 · p. 3

2.2 Cross-Session Search, Workspace Organization, and Learner-Authored Consolidation

  • Cross-session systems preserve context, but learner-authored notes also require effortful selection and organization that may support encoding while burdening users.Prior work leaves open whether learner-authored and system-generated cues differ when both are available at resumption.Source: §2.2 · pp. 3–4

2.3 Generative AI for Consolidation, Sensemaking, and User Agency

  • ICAP distinguishes passive reception from constructive generation, while cognitive-load theory suggests scaffolds may reduce extraneous effort yet work differently by learner expertise.It remains unclear whether editing generated content supports learning comparably to authoring, and agency-preserving steering may shift engagement toward active or constructive work.Source: §2.3 · pp. 4–5

2.4 Synthesis: Prior Work and Research Questions

  • Prior studies have not jointly held the search environment constant, varied consolidation origins, and measured learning after resumption; this study embeds both scaffolds in one interface.The design varies the consolidation scaffold while keeping the foundational search and workspace environment constant.Source: §2.4 · p. 5

3 System Design

The system provides a shared aggregated-search workspace with persistent, customizable clusters, then offers either learner-authored notes or steerable GenAI summaries as consolidation scaffolds. STAIS summaries are source-bounded and support user correction, reorganization, and fact-checking.

3.1 Shared Design Foundation

  • The shared workspace aggregates academic-library and Wikipedia results, organizes saved resources into customizable thematic clusters, and preserves workspace state across sessions.Resource provenance cues and a New badge support later reacquaintance with saved items.Source: §3.1 · p. 6

3.2 Note-Taking (NT)

  • NT provides rich-text editors beneath each cluster label and a global note area, leaving consolidation work to learners and preserving notes across sessions.Learners can summarize, record why resources were saved, note questions, or leave reminders.Source: §3.2 · pp. 6, 8, Figure 1 · p. 7

3.3 Steerable AI Summaries (STAIS)

  • STAIS generates summaries from saved resources at cluster or workspace scope, with user steering through reorganization, inline edits, and claim-level fact-checking.Accepted edits and fact-check corrections persist through regeneration; verdicts are advisory, so users remain responsible for judging claims.Source: §3.3 · pp. 8–9

4 Evaluation Methodology

The study compares human-authored notes with steerable GenAI summaries in a controlled, two-session search task. It assesses their use, perceived value, metacognition, cross-session support, and learning outcomes through interaction, survey, and writing evidence.

4 Evaluation Methodology

  • The controlled study examines how note-taking and steerable GenAI summaries shape exploratory search, task resumption, continued searching, and post-resumption learning.Participants completed the same unfamiliar search task in two in-person sessions one week apart.Source: §4 · p. 9

4.1 Study Design

  • Participants were block-randomized to NT or STAIS in a between-subjects experiment, with the two-session workflow and main between-condition comparisons specified in advance.Correlations, composite scores, and sensitivity checks were exploratory.Source: §4.1 · p. 10

4.2 Measures

  • The study combined interaction logs, self-reported perceptions, and a written assessment; learning measures included self-reported knowledge, interest, certainty, and confidence.The writing assessment counted accurate domain-relevant concepts and explicit connections among them.Source: §4.2 · pp. 10–11
  • Measures also covered utility, perceived value, metacognition, exploratory-search support, and task preparation and resumption.Utility included efficiency, saved-resource precision, feature use, source composition, and mixed-source clusters; the other domains used self-report instruments.Source: §4.2 · p. 10

4.3 Participants and Task

  • Thirty-six undergraduate computer science students without prior topic coursework searched for evidence for a brief on Internet-routing security, using an academic library and encyclopedia.The domain was selected for its technical vocabulary, rich conceptual structure, and suitability for screening prior familiarity.Source: §4.3 · p. 11

4.4 Procedures

  • Across two sessions one week apart, participants explored and consolidated a workspace, returned to reacquaint themselves, continued searching, and completed a writing assignment with the workspace visible.The initial session allocated 10 minutes to exploratory search and 5 minutes to consolidation.Source: §4.4 · pp. 11–12

4.5 Data Analysis

  • Baseline equivalence and survey-composite reliability were checked; comparisons used t-tests or Mann–Whitney tests as appropriate, with exact p-values and no family-wise correction.Composite scores were treated as approximately continuous; count-based outcomes used Mann–Whitney tests, and significance was set at α = 0.05.Source: §4.5 · pp. 12–13
  • Writing outcomes were independently coded for concepts and connections, while condition-specific taxonomies characterized notes and STAIS corrections without treating the taxonomies as equivalent.Hand-coded measures were independently rated, with disagreements resolved through discussion and consensus.Source: §4.5 · p. 13

5 Results

Across conditions, outcomes diverged by scaffold: STAIS was associated with faster resumption and broader conceptual coverage, while NT showed stronger inter-concept connections and self-reported certainty.

5.1 RQ1: Utility

  • In the resumed session, STAIS showed shorter reacquaintance time, higher saved-resource precision, and more items moved between clusters.Initial-session task efficiency, saved-resource precision, and general interaction patterns did not differ; inline bullet corrections increased significantly in resumed STAIS use (Z = 3.52, p < 0.001).Source: §5.1 · pp. 13–14, Table 3 · p. 14

5.1.1 Consolidation Artifacts

  • NT participants reduced lower-order activities, whereas STAIS participants added higher-order corrections while continuing to correct factual errors in generated content.Both conditions showed more cognitively demanding activity in the resumed session, but through different routes.Source: §5.1.1 · p. 14, Table 4 · p. 14

5.1.2 Source Type Composition

  • In the resumed session, STAIS participants saved more Wikipedia resources and formed more mixed-source clusters, while NT participants saved a greater proportion of library results.The authors interpret this pattern as potentially reflecting summary and fact-checking activity surfacing claims suited to reference-source verification.Source: §5.1.2 · p. 15, Table 5 · p. 15

5.2 RQ2: Perceived Value

  • After resumption, perceived usefulness, satisfaction, and the reward factor were higher for STAIS, while dimensions tied to shared interface features did not differ.No significant perceived-value differences appeared after the initial session.Source: §5.2 · p. 15, Table 6 · p. 15

5.3 RQ3: Metacognition

  • STAIS participants reported more strategy use and higher planning and evaluation, while NT participants reported higher adaptation in the resumed session.The authors suggest summaries made strategic options more visible, while note writing required learners to construct and maintain their own domain organization.Source: §5.3 · pp. 15–16, Table 7 · p. 16

5.4 RQ4: Exploratory Search

  • After resumption, NT participants reported greater support for uncertainty reduction, whereas STAIS participants reported greater support for information extraction.The authors relate these differences to the active formulation involved in note writing and the automated extraction in summaries.Source: §5.4 · p. 16, Table 8 · p. 16

5.5 RQ5: Cross-Session Support

  • Perceived support for task resumption was higher for STAIS, while perceived support for task preparation did not differ between conditions.The authors propose that ready-made summaries offered a structured entry point and lowered perceived effort when returning to the task.Source: §5.5 · p. 16, Table 9 · p. 17

5.6 RQ6: Post-Resumption Learning

  • STAIS participants had higher concept counts, while NT participants had higher connection counts; STAIS also reported greater knowledge and interest gains, whereas NT reported greater certainty.The concept-count advantage remained after adjustment for resumed-session Wikipedia saves (F(1, 33) = 10.24, p < .01, partial η2 = .237); self-reported learning generally dipped during the gap but rose to its highest measured levels after resumed search.Source: §5.6.2 · p. 17, §5.6.1 · p. 17, Table 10 · p. 17

6 Discussion

The discussion links the two scaffolds’ different allocations of consolidation work to contrasting post-resumption learning profiles. It argues that support should fit learners’ goals while noting limits on what the study can establish.

6.1 What the Two Interfaces Asked of Participants

  • NT placed authorship and organization with learners, while STAIS had learners steer, check, and edit system summaries; resumed work showed synthesis in NT notes and meaning-level revisions in STAIS.NT notes shifted toward cross-source synthesis, evaluation, and planning; STAIS revisions included factual edits and reframing or elaboration.Source: §6.1 · p. 18

6.2 Breadth and Depth of Post-Resumption Understanding

  • NT participants produced more inter-concept connections and reported higher certainty, while STAIS participants produced more concepts and reported higher knowledge and interest.The authors characterize this as deeper relational understanding with NT versus broader coverage with STAIS.Source: §6.2 · p. 18
  • The A&K taxonomy maps concept count to Conceptual/Understand, connection count to Conceptual/Analyze, and self-reported knowledge to Factual/Remember.The study gathered evidence for three of the taxonomy’s 24 knowledge-by-process cells.Source: §6.2 · p. 18

6.3 Relation to Prior Work

  • Against reports of shallower chat-style LLM learning and weaker delayed GenAI-study outcomes, integrated STAIS supported breadth while NT supported depth.STAIS was integrated into the search workspace through steerable, source-linked summaries, inline corrections, and fact-checking rather than a separate chat window.Source: §6.3 · p. 19

6.4 Design Implications

  • Choose NT for coherent integration and STAIS for broad coverage and rapid re-entry; learners’ tolerance for authoring effort may also shift across task phases.The discussion suggests offering both modes, selectable by the learner or adapted to task phase.Source: §6.4 · p. 19
  • Source-linked bullets and verdict-labelled checks appeared to make active review tractable; resumed users fact-checked more, accepted more corrections, and made more factual-error and higher-order revisions.The authors recommend presenting GenAI content as inspectable, source-linked, and easy to revise, with active checking encouraged.Source: §6.4 · p. 19

6.5 Limitations and Future Work

  • The study cannot isolate individual scaffold components; it also covers one technical domain and student population, uses one GenAI model, and lacks a manual-verification control.The authors note that these constraints limit attributing differences narrowly to agency, GenAI support, or any one component.Source: §6.5 · pp. 19–20
  • A five-minute initial consolidation window, laboratory setting, one-week interval, and open-workspace assessment constrain generalizability and do not establish unaided delayed retention.The writing task measured synthesis with access to accumulated evidence, not independent retention.Source: §6.5 · p. 20

7 Conclusion

The study finds contrasting post-resumption profiles rather than one best consolidation scaffold. It presents allocation of consolidation work as a design variable and recommends interpreting the findings within the limits of bundled scaffolds and open-workspace assessment.

7 Conclusion

  • The two scaffolds produced contrasting profiles rather than one overall best approach: STAIS showed better cross-session support, while both scaffolds had distinct post-resumption learning strengths.STAIS was generally associated with greater utility and perceived value; each condition had different strengths in metacognition and exploratory search.Source: §7 · p. 20
  • Align consolidation support with the learner’s immediate objective: broader GenAI scaffolding for conceptual mapping and rapid re-entry, and human-authored consolidation for relational integration and explanatory structuring.The conclusion positions the allocation of consolidation labour between learner and system as a consequential design variable.Source: §7 · p. 20
  • Interpret the findings within the limits of bundled scaffolds and open-workspace synthesis; future work should test component effects and add closed-book long-term retention measures.The authors also propose a unified interface combining steerable summaries and human-authored notes, then testing whether both breadth and depth can be achieved.Source: §7 · p. 20

A Implementation Details

The appendix provides technical details for the system components described in Section 3.

A Implementation Details

A.1 Summarization Pipeline

The summarization pipeline uses evidence-bounded prompts to synthesize saved resources into cited bullets. For large workspaces, it summarizes chunks before producing an attribution-preserving synthesis.

A.1 Summarization Pipeline

  • Outputs are structured JSON bullets with inline supporting-resource identifiers; large workspaces use per-chunk summaries followed by an attribution-preserving synthesis pass.Source: §A.1 · p. 20
  • GPT-4o-mini handles summarization; clustering uses temperature 0.2, while summarization and fact-checking use the API default temperature.Source: §A.1 · p. 20
  • The prompt restricts summaries to provided content and asks for cross-document agreements, contradictions, and complementary findings, while calibrating tentative claims and unknowns.Source: §A.1 · p. 21

A.2 Fact-Checking Prompt Design and Validation

The fact-checker uses calibrated, evidence-bounded prompts and per-claim labels. Validation found stronger precision for Supported verdicts than for Contradicted or Missing Evidence, and participants accepted suggestions selectively.

A.2 Fact-Checking Prompt Design and Validation

  • The prompt labels claims SUPPORTED, PARTIAL, CONTRADICTED, or MISSING and treats absent evidence as distinct from contradiction.Source: §A.2 · p. 21
  • Supported-verdict precision was 0.91, versus 0.68 for Contradicted and 0.63 for Missing Evidence; inter-rater agreement was κ = 0.78.Source: §A.2 · p. 21
  • The false-positive rate was 14% and the false-negative rate was 22%; participants accepted fact-check suggestions in roughly 29–36% of activation events.Source: §A.2 · p. 21
  • Per-category recall was not computed because stratified sampling used 15 verdicts per category; the authors recommend a larger randomly sampled evaluation for precision and recall.Source: §A.2 · p. 21

A.3 Correction Record Persistence

Inline corrections are stored for each workspace and applied in subsequent summary regenerations.

A.3 Correction Record Persistence

  • Each inline edit and its supporting source identifier are stored as a workspace-scoped correction record.Source: §A.3 · p. 21
  • On later regenerations, saved corrections instruct the model to use the corrected wording verbatim and exclude the original phrasing.Source: §A.3 · p. 21
  • Edits accumulate over time and persistently shape future outputs without requiring users to re-enter them.Source: §A.3 · p. 21

A.4 Conversational Context

The system maintains conversation history for each user-workspace pair, with a stateless fallback if the API is unavailable.

A.4 Conversational Context

  • A persistent thread is created on first access for each user-workspace pair, and its identifier is stored locally.Source: §A.4 · p. 21
  • Subsequent requests reuse the thread so the model retains prior summaries, corrections, and topic framing.Source: §A.4 · p. 21
  • If the API is unavailable, stateless mode retransmits the full document content and correction history with each request.Source: §A.4 · p. 21

A.5 Workspace Snapshot Schema

Each save stores a complete workspace snapshot, and the latest snapshot restores the workspace as it was left.

A.5 Workspace Snapshot Schema

  • Each save stores cluster assignments, labels, summaries, an outline, notes, reminders, and saved resources with provenance in a complete snapshot.Snapshots are stored in MongoDB by user and workspace; the full-snapshot strategy restores the latest state, including corrections and reorganizations.Source: §A.5 · p. 21

Declaration of generative AI and AI-assisted technologies in the writing process

  • The first author used generative AI for early-draft text editing and clarity, then reviewed and edited the content; the second author revised independently without AI.The authors take full responsibility for the publication's content.Source: Declaration of generative AI and… · p. 22

CRediT authorship contribution statement

  • Momeni contributed across conceptualization, methods, software, analysis, investigation, data curation, validation, visualization, and writing; Hoeber contributed conceptualization, methods, supervision, validation, writing, administration, and funding.Source: CRediT authorship contribution statement · p. 22
Loading 2610.12755v1…