Source-linked AI summary
The Living Library: Transforming Archival Collections into Conversational Knowledge Systems -- Lessons from the Theodore Roosevelt Presidential Library
Pengce Wang, Lucia Ronchi Darre, Matt Briney, Michaell Bakalars, Dan Rutkowski, Ursula Hardy, David Wolf, Laura Hoffman, Allen Kim, Shawn Wright, Juan Lavista Ferres
TL;DR
Fragmented, partially cataloged archives are difficult to make conversationally accessible without sacrificing historical integrity. The paper presents the Living Library, a governed four-layer framework deployed at the Theodore Roosevelt Presidential Library, and reports Talk to TR as a public-exhibit experience rather than a controlled evaluation. Its design combines grounded analogical retrieval, streaming interaction, and autonomous museum operation, while leaving faithfulness, anachronism, and experiential presence open for further study.
Problem
Fragmented archival collections remain difficult to access, search, and interpret broadly, raising the question of how to improve accessibility without sacrificing historical integrity.
Method
The Living Library combines corpus creation, AI processing, retrieval and reasoning, and an optional conversational interface, with OCR, curatorial review, hybrid indexing, and grounded exhibit interaction.
Results
The deployed system unified approximately 300,000 records into a governed corpus and supported Talk to TR with analogical grounding, dual-path retrieval, streaming, and museum-scale operation.
Takeaways & Limitations
The deployment offers a transferable model for making archival collections conversationally accessible, with or without an embodied conversational layer.
Takeaways & Limitations
Evidence is largely observational rather than controlled, and OCR errors, latency, retrieval-routing errors, and unanswered questions about faithfulness, anachronism, and experiential presence remain.
Abstract
from arXiv · showhide
We present the Living Library, an end-to-end framework for transforming fragmented digital archives into governed, conversational, in-person exhibit experiences. Developed and deployed at the Theodore Roosevelt Presidential Library, the framework comprises four layers: digitization and corpus creation, AI-powered processing, retrieval and reasoning, and an optional embodied conversational interface. The first three layers aggregate a 300,000-record collection, apply OCR and structured metadata enrichment for expert curatorial review, and publish records to a hybrid dense/semantic index. Expert review is conducted through the Archivist App, a curator-facing interface that supports correction of AI-generated transcriptions and metadata. The governed corpus powers both a researcher-facing interface and Talk to TR, a continuously operating exhibit that embodies Theodore Roosevelt as a full-scale digital human within a museum environment. To support live, face-to-face interactions, Cross-Era Analogical Grounding reframes contemporary questions through documented historical parallels, allowing Roosevelt to address present-day topics without inventing facts. Dual-path retrieval and end-to-end streaming keep responses grounded and responsive. Layered watchdogs, visitor-session isolation, automated conversation management, and independently restartable services enable reliable unattended operation for hundreds of visitors. Avatar realism, spatial audio, lighting, staging, and conversational design are developed and evaluated as an integrated experience. Rather than report a controlled benchmark, we describe lessons from operating Talk to TR as a public exhibit and offer a transferable model for transforming archival collections into believable, in-person conversational experiences.
1 Introduction
The paper asks how fragmented, partially cataloged archives can become conversationally accessible without sacrificing historical integrity. It presents the Living Library and Talk to TR as an end-to-end response combining governed archival data, grounded retrieval, and an embodied exhibit.
- The Living Library combines digitization, AI processing, retrieval and reasoning, and an optional conversational interface into a governed archival knowledge system.
- Talk to TR renders Theodore Roosevelt as a full-scale digital human who converses face to face using historically verifiable sources.
- The framework includes curator review, physical-digital staging, replicability principles, and deployment observations rather than a controlled benchmark.
- Cross-Era Analogical Grounding reframes contemporary questions as historically attested analogs, enabling modern-topic engagement without fabricating historical claims.
- Dual-path retrieval and streaming address the latency demands of real-time embodied conversation while keeping responses grounded in archival material.
- The exhibit supports unattended public operation through session isolation, automated lifecycle management, watchdog recovery, and bounded context management.
2 Background: Traditional Archival Practice
Traditional archival collections are fragmented, inconsistently cataloged, and difficult to access, leaving substantial material undiscovered or unread. The Living Library addresses this combined problem by making collections accessible, searchable, and interpretable while preserving historical integrity.
- Fragmented collections: Records are dispersed across repositories, with TRPL holdings spanning more than forty public and institutional sources.
- Manual cataloging limitations: Item-level cataloging is time-intensive and produces inconsistent vocabulary, granularity, and completeness across institutional eras.
- Access constraints: Traditional archival engagement requires physical access, specialized training, or institutional affiliation, despite digitization efforts.
- Underutilized assets: These barriers leave large portions of collections unprocessed, undiscovered, or unread as acquisition-driven backlogs grow.
- The resulting challenge is to make vast, fragmented, partially cataloged collections universally accessible, searchable, and interpretable without sacrificing historical integrity.
3 Related Work
Related work spans persona agents, cultural-heritage installations, retrieval-augmented systems, streaming embodied agents, conversational memory, archive digitization, and safety. Talk to TR combines these traditions while grounding a real historical persona in primary sources under face-to-face constraints.
- Persona and role-play language agents: Persona agents commonly encode characters in model parameters or static prompts, whereas Talk to TR grounds its persona non-parametrically in the subject’s primary sources.
- Interactive cultural-heritage and historical-figure installations: Cultural-heritage systems often replay authored or recorded human utterances, while Talk to TR generates answers for a historical figure who left no recordings.
- Retrieval-augmented and active retrieval: Talk to TR extends active retrieval with asynchronous next-turn prefetching and verbal-filler masking for hard real-time embodied interaction.
- Real-time, streaming, and embodied conversational agents: Unlike prior streaming and embodied-agent work, the system reports end-to-end first-token latency including grounding costs.
- Long-term memory in conversational agents: The system uses short-horizon per-turn and per-visit memory but does not maintain visitor-linked personal memory across separate visits.
- Historical document digitization and archive access: Its governed OCR and enrichment pipeline treats residual digitization errors as downstream concerns and exposes the reviewed corpus through a grounded conversational persona.
- Faithfulness, attribution, safety, and guardrails: A layered, fail-open safety stack addresses public-kiosk guardrails while treating post-lifetime references as a distinctive historical-persona faithfulness concern.
4 The Living Library Model
The Living Library separates archival corpus construction from retrieval and optional embodied conversation. At TRPL, the governed corpus supports both Campfire and Talk to TR through layered services and a physical exhibit.
- Corpus construction: TRPL’s pipeline preserves institution-controlled records, applies OCR and metadata extraction, and publishes chunked embeddings to a hybrid search index.Hard metadata remains preserved while AI-generated text and soft metadata are added downstream.
- Retrieval and reasoning: Layer 3 retrieves relevant material from the governed corpus so an LLM can compose grounded, attributable responses rather than rely on parametric memory.Talk to TR additionally uses Cross-Era Analogical Grounding and latency-aware dual-path retrieval.
- Architecture: The framework decomposes the deployment into corpus creation, AI processing, retrieval and reasoning, and an optional embodied interface.Layers 1–3 support Campfire; Layer 4 adds Talk to TR’s voice, avatar, and physical presence.
- Embodied interface: Talk to TR adds a full-scale digital human and physical presentation, integrating staging, lighting, spatial audio, visitor positioning, and conversational design.The exhibit is designed as a believable in-room encounter rather than a web chatbot with an avatar.
5 Technical Architecture
The technical architecture combines source-grounded OCR evaluation, curator-controlled indexing, modular dialogue services, and historical retrieval strategies. Its design joins factual grounding with latency-aware streaming and operational flexibility.
- OCR and review: OCR candidates are evaluated against reference text using CER, WER, BLEU, and embedding cosine on a stratified sample of handwritten and typed material.The semantic metric reflects the downstream retrieval-grounded dialogue task.
- OCR and review: GPT-5 leads every OCR metric, but GPT-4.1 is the production default because it narrowly trails GPT-5 while balancing quality, cost, and throughput.Embedding cosine is at least 0.94 for the strongest models, while handwritten material remains the dominant error source.
- OCR and review: The Archivist App lets curators inspect and correct AI-generated transcriptions and metadata while preserving immutable source metadata and enabling continuous publication, correction, and withdrawal.This makes the 300,000-record backlog searchable immediately while quality improves through review.
- Dialogue service: The dialogue service uses independently bounded components for visitor sensing, orchestration, streaming speech, retrieval, avatar delivery, and rendering.A structured turn object combines the spoken response with routing, memory, completion, and retrieval-control signals in one LLM call.
- Historical grounding: Cross-Era Analogical Grounding reframes contemporary questions as historically attested analogs, allowing modern topics to receive in-period, attributable answers.The design bridges modern queries to century-old records without direct archival references for those topics.
5.5 Meeting the Real-Time Budget
The exhibit meets face-to-face responsiveness and reliability demands through streaming, dual-path retrieval, bounded context, non-blocking safeguards, and autonomous recovery. Deployment logs provide an operational latency profile rather than a controlled benchmark.
- Latency: Local streaming ASR, asynchronous prefetch, synchronous self-routing, adaptive fillers, and end-to-end streaming reduce perceived response delay.The pipeline overlaps ASR, LLM generation, TTS, and avatar frames instead of summing their full durations.
- Latency: 2.80 s average TTFT was measured across 457 completed answers, with 97% under 5 s and 69% under 3 s.The deployment logged 653 push-to-talk releases; 156 were superseded by interruption or re-asking, while smaller groups produced repeats or timeouts.
- Context management: Context management isolates visitors with moving watermarks, bounded cross-phase bleed, and in-place refresh while retaining only visit-coherent state.Refresh occurs after at least 180 seconds of microphone idle time, about 800 history entries, or about 8 hours of session runtime.
- Safety: The safety design uses progressively heavier, non-blocking checks so detection can occur later rather than stall an utterance in progress.Deployment controls add persona boundaries, fallback responses, escalation, and curator-reviewed grounding constraints.
- Autonomous operation: Watchdogs reconnect, restart, or degrade failed services, while a warm session and independently restartable components let the exhibit serve hundreds of visitors daily with little staff attention.The visitor lifecycle resets in place rather than rebuilding the system between visits.
6 Design Principles and Governance
The Living Library treats governance, source traceability, human curation, configurable infrastructure, and embodied experience as interdependent design principles. It also recognizes attribution, archival bias, fabrication, and privacy as persistent constraints.
- Source-grounded outputs: Every user-facing response is grounded in verified primary sources, historical analogies, and continuing expert correction.The persona is grounded non-parametrically, while Cross-Era Analogical Grounding addresses modern questions through attested historical parallels.
- Human-in-the-loop governance: Automation accelerates processing, but curators retain authority over record quality, escalation, and experience boundaries.Records can enter the index with review status and AI provenance, then be verified and corrected through the Archivist App.
- Scalable by design: Vendor- and model-pluggable services, independently restartable components, and runtime-overridable prompts support adaptation across collections and model generations.The architecture avoids hard-wiring the production OCR model, real-time services, or persona prompts to a single deployment configuration.
- Embodied experience: Immersion depends on avatar realism, display fidelity, lighting, audio, staging, and strict real-time performance alongside language-model grounding.Deployment observations led the system to treat embodied presence and latency as first-class design concerns.
- Historical integrity: Historical attribution is a first-class constraint because generated speech risks attributing statements to Roosevelt that he never made.Archival grounding, persona and chronology controls, and visitor disclosure are intended to keep generated responses close to the documentary record.
- Residual concerns: The deployment retains residual concerns about archival bias, imperfect boundaries between inference and fabrication, and privacy governance for presence and speech data.Curatorial review and historical framing can mitigate but not eliminate omissions and bias, while source-grounded analogies remain novel utterances.
7 Ethics, Privacy, and Data Governance
The exhibit is designed for anonymous, opt-in public use with limited data collection, defined retention, restricted access, and explicit visitor notice. Research involving visitor-level data requires institutional ethics review or an exemption determination.
- Data minimization: The exhibit minimizes data collection, avoids intentionally linking interactions to identities, and applies defined retention and access controls.It is designed for anonymous walk-up use in a public library setting.
- Notice and participation: Visitors receive notice about AI-generated personas, presence sensing, speech transcription, and operational-log retention before participating.Speech processing begins only after pressing push-to-talk, and visitors may end interactions at any time.
- Children and vulnerable visitors: The system does not request personal information from children or vulnerable visitors, while minor supervision follows on-site policy and signage.The exhibit is intended for a general audience.
- Presence sensing and RFID: Presence sensing emits only a transient non-biometric identifier and does not retain facial or biometric data; RFID supports staff control rather than visitor identification.The transient tracking identifier lasts approximately 30 seconds.
- Audio and operational logs: Audio is processed as a transient ASR stream, while operational logs contain randomized session identifiers, transcripts, model data, retrieval metadata, safety verdicts, and health measurements.The logs do not create identity fields or intentionally associate interactions with real-world identities.
- Retention and access: Operational logs are retained for up to 30 days, with earlier deletion requests possible when an interaction can be identified by approximate date and time.Access is limited to authorized operators and maintainers for troubleshooting, safety auditing, and approved evaluation.
- Ethics review: The reported results use aggregate telemetry and exclude identifiable visitor interactions; research using visitor-level data requires prior ethics review or a formal exemption.The requirement follows applicable institutional policy.
8 Replicability Framework
The replicability framework separates collection readiness, corpus construction, AI processing, retrieval, conversation, and optional avatar deployment into staged steps. Steps 1–4 form the broadly transferable core, while embodied conversation adds specialized real-time and persona-design demands.
- Step 1: Assess collection readiness: Replication begins by inventorying fragmented repositories and digitization gaps, with the readiness assessment determining the scope of subsequent work.This step is policy and inventory work rather than engineering.
- Step 2: Build the corpus: Layer 1 digitizes and aggregates materials into institution-controlled storage before AI processing, with governance for identifiers, rights, and metadata schemas.The schema separates immutable hard metadata from later enrichment.
- Step 3: Apply AI processing: Layer 2 applies model-pluggable OCR and metadata enrichment, validates model choice on a stratified source-grounded sample, and plans expert review.The evaluation framework supports selecting the production model rather than hard-wiring one choice.
- Step 4: Implement retrieval: Layer 3 indexes the corpus for hybrid dense/semantic search and exposes it through a governed retrieval contract.This layer alone can provide a researcher-facing tool without a conversational interface.
- Step 5: Add conversation: The conversational layer connects an LLM to retrieval and defines persona grounding, analogical reframing, and non-blocking safety patterns.These interaction patterns extend the governed retrieval system rather than replace it.
- Step 6: Avatar experience: The optional avatar layer adds persona and narrative design plus voice and visual rendering under real-time and autonomous-operation constraints.It is intended for institutions seeking an embodied presence rather than text- or voice-only assistance.
9 Impact and Outcomes
Layers 1–3 make fragmented collections consumable through a single governed, semantically searchable contract, while Layer 4 turns that corpus into an embodied encounter. Public deployment reached close to 5,000 visitors in its first two weeks, but observational logging cannot establish faithfulness, non-anachronism, or experiential presence to the standard of a dedicated study.
- Layers 1–3: Layers 1–3 unlock previously inaccessible or underprocessed assets through one governed contract, without requiring an eventual conversational-avatar deployment.Institutions can add Layer 4 later against the same corpus.
- Researcher access: A unified semantic index shortens the path from questions to relevant primary sources across repositories and document types.The paper presents this as a structural capability, not a measured time-to-insight outcome.
- Public deployment: Close to 5,000 visitors engaged Talk to TR during its first two weeks of public operation.Layer 4 changes the governed corpus from a search task into an encounter, while the reported evidence remains observational.
- Evidence boundary: The deployment does not establish whether the encounter is faithful, free of anachronism, or experientially present to the standard of a dedicated study.The authors explicitly avoid overstating what two weeks of observational logging can show.
10 Limitations and Future Work
The paper identifies deployment, evaluation, and generalization boundaries that remain unresolved for the Living Library. It frames these as directions for future work rather than reported results.
- Scope and generalization: The study covers one historical figure and one archive, limiting evidence for generalization to other institutions and collections.Broader deployment raises unresolved questions about metadata standardization, model governance, responsible representation, and educational measurement.
- Evaluation boundaries: The deployment report does not provide controlled measurements of faithfulness, anachronism, experiential presence, or autonomous-operation reliability.These claims currently rest largely on deployment observation rather than controlled evaluation.
- System limitations: OCR errors are reduced through expert review but can remain and propagate into retrieval and historical faithfulness.The system may also retrieve unnecessarily or omit retrieval when it is needed.
- Future evaluation: Long-term reliability and changing model behavior remain open governance questions for applying the framework beyond this deployment.The paper specifically calls for longitudinal reliability logging and practices for maintaining accuracy as underlying models change.
- Future evaluation: Assessing faithfulness and anachronism requires systematic response annotation against sources, while experiential presence requires a dedicated visitor study.The paper identifies these as future-work requirements rather than results established by the exhibit.
11 Conclusion
The Living Library concludes that fragmented archives can be transformed into governed, conversational knowledge systems through a layered framework. Its Talk to TR deployment combines grounded historical dialogue with real-time and museum-scale operational design, while remaining an experience report rather than a controlled study.
- Framework: The four-layer framework combines corpus creation, AI processing, retrieval and reasoning, and an optional conversational interface.The framework is presented as replicable and instantiated at the Theodore Roosevelt Presidential Library.
- Framework: Layers 1–3 unify a ∼300,000-record collection into a searchable corpus reviewed through the Archivist App and consumed by Campfire.The governed corpus also supplies the conversational Layer 4 system.
- Talk to TR: Talk to TR grounds a conversational digital human in the shared corpus while balancing faithfulness, non-anachronism, and real-time engagement.Cross-era analogical grounding addresses contemporary topics through historically attested analogs.
- Talk to TR: Dual-path retrieval, end-to-end streaming, dynamic context management, and fail-open safety support responsive, all-day multi-visitor operation.Layered watchdogs, session isolation, and automated lifecycle management support unattended exhibit operation.
- Scope and future work: The paper presents the Layer 4 deployment as an experience report and leaves faithfulness, anachronism, reliability, and experiential presence open for future study.The proposed design principles and replicability framework are intended to transfer beyond the specific deployment.