PaperlayerView exact revision on arXiv

Source-linked AI summary

Structured State Reconciliation for Human-AI Task Handover

Kayleigh Bishop, Maria P. Stull, Breanne Crockett, Bradley Hayes

arXiv:2608.28907v1cs.HCcs.AI

TL;DR

Task handover requires reconstructing current state from complementary but incomplete and potentially conflicting system records and human observations. The paper introduces provenance-aware structured reconciliation that aligns both sources, detects conflicts, preserves provenance, and renders constrained reports. In 13 controlled paired task states, reconciliation improved estimated task-state utility over either source alone, reduced misinformation versus direct LLM synthesis at comparable utility, and made task-aware rendering more efficient than exhaustive rendering.

  • Problem

    Task handover requires combining precise but partial system records with human reports that contain uninstrumented knowledge yet remain vulnerable to omission and memory error.

  • Method

    A provenance-aware pipeline converts telemetry and human reports into a shared typed representation, aligns and reconciles facts, detects conflicts, preserves provenance, and constrains report generation.

  • Results

    In 13 controlled paired task states, reconciliation preserved greater estimated task-state utility than either source alone and maintained comparable utility with substantially less misinformation than direct end-to-end LLM synthesis.

  • Takeaways & Limitations

    Provenance-aware state reconciliation is supported as an auditable design pattern for safer AI-assisted handover, while human reports retain strategic knowledge beyond state-focused metrics.

  • Takeaways & Limitations

    The study evaluates reports and cost-model estimates rather than observing downstream recipient task performance, within a controlled environment with known ground truth.

Abstract

from arXiv · show

Task handover requires communicating enough current state for a successor to resume work, yet the relevant information is often divided between system records and human observations. System records can be precise and timestamped but only partially observe the task, while human reports capture intent and task knowledge that no log contains but are vulnerable to omission and memory error. We present a provenance-aware pipeline that converts task telemetry and human-authored reports into a shared typed task-state representation, aligns and reconciles their facts, detects conflicts, and generates structured handover reports. We evaluate the approach on 13 paired task states collected in a controlled spatial multitask environment, using task-grounded metrics that estimate the state-reconstruction cost a report would spare a hypothetical recipient and the misinformation burden it would impose. Reconciling both sources preserved greater estimated task-state utility than either the user report or telemetry alone. Relative to a direct end-to-end LLM given the same inputs, structured reconciliation maintained comparable estimated utility while incurring substantially less misinformation, and task-aware rendering retained utility more efficiently (per token) than exhaustive rendering. An exploratory content analysis further shows that human reports contain substantial strategic knowledge that lies outside state-focused metrics. These results support provenance-aware state reconciliation as a design pattern for safer AI-assisted handover.

1 Introduction

Task handover must help a successor reconstruct current task state from incomplete, potentially conflicting system and human accounts. The paper presents provenance-aware reconciliation to combine these sources, control misinformation, and evaluate reports by recoverable task-state utility.

  • Motivation: Incoming agents must reconstruct a sufficiently accurate current task-state model from incomplete externalization of the outgoing agent’s awareness.Reports may serve as the sole external scaffold for situation-awareness reconstruction during handover.
  • Motivation: System records are precise and timestamped but omit uninstrumented context, while human reports add strategy and anticipated steps but risk omission and memory error.Effective assistance must reconcile both partial accounts rather than summarize whichever source is available.
  • Motivation: Direct LLM synthesis leaves source reliability, contradictions, stale facts, unsupported content, and provenance implicit.Free-form summaries cannot distinguish corroborated facts from memory-only claims or active source disputes.
  • Approach: The proposed pipeline converts telemetry and free-text reports into a shared typed representation, aligns facts, detects conflicts, reconciles temporal state, and preserves provenance.A constrained LLM renders the reconciled state; task-aware rendering foregrounds outstanding needs and required locations, while exhaustive rendering reports all available state.
  • Evaluation: The evaluation uses 13 paired task states and estimates the state-reconstruction effort a report spares a recipient alongside its misinformation burden.Reports are scored against ground-truth task state under an explicit search-cost model rather than by observing recipients resuming tasks.
  • Findings: Reconciling both sources preserved more estimated task-state utility than either alone, reduced misinformation versus direct end-to-end LLM synthesis at comparable utility, and improved the utility–length tradeoff over exhaustive rendering.Exploratory analysis also found substantial strategic knowledge in human reports that state-focused metrics do not credit.

2 Background and Related Work

Handover research frames transfer as collaborative, context-dependent information work across distributed situation awareness. This paper positions provenance-aware reconciliation as a way to integrate partial system and human accounts while prioritizing task-relevant state.

  • Handover as collaborative knowledge transfer: Handover sheets scaffold knowledge transfer, while their absence can cause repeated work, lost productivity, and loss of shared procedural knowledge.
  • Handover as collaborative knowledge transfer: Information work includes seeking, receiving, sharing, synthesizing, and interpreting information needed for task continuity.
  • Situation awareness and information prioritization: Distributed situation awareness requires communicating environmental state, its significance for current goals, and likely future trajectories.
  • Provenance-aware reconciliation: The evaluation distinguishes state-focused information from procedural knowledge and measures the information work saved through handover content.
  • Situation awareness and information prioritization: Principled prioritization is possible because not all information is equally consequential for reconstructing situation awareness after handover.
  • Provenance-aware reconciliation: The paper treats system telemetry and human reports as complementary, potentially conflicting models whose facts require provenance preservation and explicit reconciliation.

3 Provenance-Aware Task-State Reconciliation

The paper models handover as provenance-aware state reconciliation: telemetry and human reports become typed facts, are aligned and compared, and are merged into a current state estimate before constrained report generation.

  • Formulation and representation: Each candidate fact retains provenance, temporal position when available, and cross-source status such as corroborated, single-source, or conflicting.
  • Formulation and representation: The pipeline combines timestamped system observations with a free-text outgoing-worker report in a unified knowledge representation.
  • Typed task-state representation: Typed task-state facts represent entities, locations, controlled task relations, and unresolved spatial references with named or existential arguments.
  • Source extraction: Telemetry is parsed deterministically from timestamped events, while user notes undergo LLM normalization followed by deterministic DSL parsing.
  • Alignment and reconciliation: Alignment matches entities and facts across graphs, classifying report content as confirmed, novel, or conflicting when incompatible values occur.
  • Alignment and reconciliation: Conflicting facts remain flagged rather than silently collapsed, while temporal reconciliation prunes superseded events to produce the current outstanding task state.

4 Data and Controlled Task

The study evaluates handover in a controlled spatial multitask environment designed to create concurrent obligations, search costs, interruptions, and complementary observability. Participants play, write reports, and provide paired states for controlled comparisons.

  • Controlled task environment: The environment models incomplete progress, concurrent obligations, spatially distributed resources, interruption, partial observability, and transfer to a successor.
  • Scope of the environment: The environment deliberately excludes clinical expertise, clinical risk, shared professional schemas, interactive clarification, and longitudinal shifts.
  • Controlled task environment: Participants complete sequential tasks for five patients across rooms, requiring item retrieval, NPC interactions, navigation, and handoff of unfinished work.
  • Controlled task environment: The task ensures outstanding work remains for handoff because participants cannot complete all possible tasks within the time limit.
  • Data collection: The study collected 13 independent task states and generated 65 paired reports, supporting within-state comparison rather than population-level behavioral inference.
  • Data collection: Participants played for 10 minutes, completed a 2-minute distractor task halfway through, and then had 5 minutes to write a handover report.
  • Observability design: Telemetry records discrete events, whereas visual information and dialogue are unavailable to the system unless included in the human report.

5.1 Evaluation scope and estimand

The evaluation measures report-level task-state utility under an explicit search-cost model rather than observing recipient behavior. Its estimand is an objectively verifiable proxy for task-resumption readiness.

  • Evaluation scope: The study evaluates whether reports contain task-state information expected to reduce reconstruction effort, without observing a receiving participant.
  • Evaluation scope: Metrics estimate information available to a hypothetical recipient using search costs derived from the controlled environment’s topology and task rules.
  • Evaluation scope: The resulting quantities are artifact-level proxies, not measurements of recipient behavior, resumption time, error rate, or task completion.
  • Evaluation scope: Objective scoring verifies report claims against known ground-truth task state rather than rater judgment.

5.2 Estimating task-state utility via information access cost

The paper estimates handover value by modeling how reported facts reduce a recipient’s task-grounded reconstruction cost, while separately penalizing misinformation. The framework extracts and aligns report facts with ground-truth state, scores location, need, and resource information, and controls for extraction and formatting concerns.

  • Evaluation controls: LLM-assisted extraction was validated with human co-annotation, while deterministic scoring against ground-truth state prevented model judgment from entering utility or misinformation calculations.The main machine-generated contrasts use a shared rigid output template, limiting differential parseability as an explanation for results.
  • Metric framework: Information Access Cost estimates how much search effort a downstream agent would avoid or waste when acting on reported knowledge.The score uses expected physical movement and interaction costs, expressed in task-derived cost units rather than literal steps.
  • Metric framework: The evaluation separates estimated task-state utility, defined as modeled reconstruction cost saved, from misinformation burden caused by incorrect reported facts.Incorrect facts can force wasted actions or searches in wrong areas.
  • Component scoring: Each target patient is evaluated through orthogonal patient, need, and resource components, with predicted facts compared independently against ground-truth state.Need captures the required medication or message-routing task, while resource captures where the needed medication or delivery target is located.
  • Location scoring: Location facts receive FULL, PARTIAL, or NONE credit according to whether they identify the true room, reduce the search space, or fail to match.The system selects the most specific fact when multiple facts reference the same entity and computes partial credit from search-space reduction.
  • Fact alignment: The pipeline resolves named and existential references, evaluates need predicates and resource specificity, and credits correct omissions when no outstanding information exists.Generic resources can receive partial rather than full credit, while facts outside the task-defined target universe are excluded from omission scoring.

5.3 Task-value density and report length

The paper evaluates task-value density as estimated task-state utility per token, while warning that the ratio is not information-theoretic compression and can favor overly short reports. Density is therefore interpreted alongside absolute utility, misinformation, and raw length.

  • Density measure: Task-value density measures estimated task-state utility conveyed per token of report text.Token counts use the o200k_base tokenizer associated with the GPT-4.1 model family.
  • Interpretation: The density ratio is not information-theoretic compression and may reward degenerately short outputs.The paper therefore reports raw report lengths and interprets density with absolute utility and misinformation results.

5.4 Characterizing information content of user reports

The content analysis distinguishes factual state transfer from strategic knowledge, meta-information, and ambiguous clauses in user reports. It treats the analysis as exploratory and descriptive, with human-validated annotation supporting the reported category observations.

  • Content categories: User-report clauses are categorized as State Transfer, Knowledge Transfer, Meta/Other, or Ambiguous/Mixed.State Transfer covers falsifiable current-state claims, whereas Knowledge Transfer includes strategy, priorities, causal knowledge, warnings, predictions, and ruled-out dead ends.
  • Knowledge Transfer: Knowledge Transfer captures information a fully state-visible agent would not automatically know, including strategy, priorities, warnings, and predictions.This category separates strategic task knowledge from factual state reconstruction.
  • Annotation validation: LLM annotation was human-checked on 30 percent of reports, achieving Cohen’s κ=.64 overall and .84/.74 for state and knowledge categorization.The paper reports these agreements to support the reliability of the exploratory observations.
  • Scope: The analysis is exploratory and descriptive, and none of the paper’s primary claims were annotated using this schema.Its observations therefore characterize report content rather than provide the basis for the primary evaluation claims.

5.5 LLM configuration and reproducibility

The pipeline uses LLMs at four specified stages while keeping subsequent transformations deterministic given their outputs. Prompts, schemas, examples, alias tables, and parsing rules are released to support reproducibility.

  • LLM stages: LLMs are used for report normalization, existential-reference resolution, report generation, and evaluation-time fact extraction or content annotation.GPT-4.1-mini is used for annotation; other transformations are deterministic given LLM outputs.
  • Reproducibility: Supplementary materials release prompts, few-shot examples, output schemas, alias tables, and parsing rules for each LLM stage.These materials are intended to make the multi-stage pipeline inspectable and reproducible.
  • Configuration: LLM calls used the OpenAI ChatCompletions endpoint with gpt-4.1-mini, zero temperature, no retry policy, and no manual output repair.The recorded knowledge cutoff was 01-Jun-2024 and access date was 18-Jul-2026.
  • Reported metrics: Table 4 reports median [range] descriptive cost metrics across 13 participants in task-derived search-cost units.Combined cost uses α=3 descriptively, while estimated cost saved and misinformation cost are reported separately as primary inferential metrics.

6 Results

Across 13 paired task states, reconciling human reports with telemetry preserved more estimated task-state utility than either source alone. Structured reconciliation maintained comparable estimated utility to end-to-end synthesis while reducing misinformation and improving utility per token; human reports also contained strategic knowledge outside the state-focused metrics.

  • RQ1: Source complementarity: χ2(4, N= 13) = 33.60, p< .001, Kendall’s W = 0.65 for estimated cost saved; weighted misinformation cost and task-value density were also significant.Weighted misinformation cost was χ2(4, N= 13) = 31.95, p< .001, Kendall’s W = 0.61; task-value density was χ2(4, N= 13) = 29.46, p< .001, Kendall’s W = 0.57.
  • RQ1: Source complementarity: Reconciling telemetry and user reports preserved more estimated task-state utility than either source alone.The two sources contributed complementary state information under the study’s cost model, not necessarily recipient performance.
  • RQ2: Structured versus end-to-end synthesis: Task-aware and end-to-end reports had comparable estimated cost saved, p_adj= .33, but task-aware misinformation was lower: Mdn = 45.67 versus Mdn = 180.0, p_adj= .004, r= .93.Task-aware reports also had higher estimated utility per token: Mdn = 2.64 versus 2.30, p_adj= .01, r= .87.
  • Exploratory content analysis: Human reports contained substantial strategic knowledge: knowledge-transfer content averaged 35 percent of report clauses, compared with 56 percent for state-transfer content.Strategic content varied from 0–66 percent across participants and was not credited by the state-focused metrics.

7 Discussion

The discussion frames handover as reconciliation of partial, differently reliable accounts before communication. Structured reconciliation primarily improved error control, while human input expanded observability but added uncertainty and strategic knowledge that the current metrics do not value.

  • Structured state reconciliation as a reliability pattern: Handover is a state-reconciliation problem: partial, differently reliable accounts must be integrated into one defensible current-state description.The system makes this integration prerequisite explicit before communicating handover information.
  • Structured reconciliation and error control: With identical inputs, structured reconciliation matched end-to-end estimated state utility while incurring less misinformation and conveying utility more densely.The discussion characterizes structure’s benefit on these data as primarily error control rather than coverage.
  • Structured reconciliation and error control: Typed facts, provenance, temporal status, and inspectable conflicts are the intermediate representation’s enforceable properties, not properties unique to one representation.Explicitly identifying conflicts makes them auditable rather than silently resolved.
  • Complementary source roles: Human input expands observability but adds uncertainty: telemetry recovered less utility because some state was unobservable, while human reports dominated surviving misinformation.The proposed division of labor is for systems to reconstruct corroborated state while humans supply intent, strategy, causal knowledge, and warnings.
  • Limitations and future work: The controlled, simplified task lacked professional expertise, real stakes, shared schemas, interactive clarification, and longitudinal shifts, limiting generalization.Task novelty may have inflated the value participants placed on experiential guidance, and real-world telemetry may be stale, missing, or wrong.
  • Limitations and future work: The state-focused metrics do not evaluate the downstream utility of the substantial strategic knowledge carried by human reports.Future work includes preserving human-only strategy alongside verified state and surfacing provenance and conflict records.

8 Conclusion

In a controlled multitask environment, reconciling worker reports with system telemetry produced more useful handover reports than either source alone. Structured reconciliation reduced misinformation at comparable utility, while task-aware rendering improved the utility–length tradeoff; human reports also contributed strategic knowledge beyond state-focused metrics.

  • Reconciled worker reports and system telemetry produced greater estimated task-state utility than either source alone.
  • Explicit structured reconciliation maintained comparable estimated utility while incurring substantially less misinformation than direct end-to-end LLM synthesis.
  • Task-aware realization improved the utility–length tradeoff relative to exhaustive reporting.
  • Human-authored reports contained substantial strategic knowledge that state-focused pipelines and metrics do not capture.
  • These findings support provenance-aware state reconciliation as an auditable design pattern for safer AI-assisted handover.
  • The reported benefits remain artifact-level estimates, making validation with task-resuming recipients a natural next step.
Loading 2608.28907v1…