PaperlayerView exact revision on arXiv

Source-linked AI summary

What Happens During Autonomous Deep Research After the User Steps Away?

Yimin Liu, Yijia Zhang, Yanmin Li, Tangwen Luo, Yuze Li, Ziling Yao, Zhi Yang

arXiv:2609.33509v1cs.AI

TL;DR

The paper asks how initial user information shapes autonomous research after the user steps away, and how intermediate actions relate to final recommendations. DRaligned compares matched conditions differing in one task-relevant factor and reads requests, drafts, and reports directionally. It finds that user-specific recommendations can emerge from largely shared research, and that coarse recommendation direction often persists through rewriting.

  • Problem

    It is unclear how initially supplied user information appears in autonomous research behavior and relates to final recommendations after human input stops.

    Source: §1 · pp. 1–2

  • Method

    DRaligned varies one task-relevant user factor in matched conditions and uses source-grounded directional readings of acquisition, workspace, and finalization behavior.

    Source: §1 · p. 2, §3.2 · p. 3, §4.3 · p. 5

  • Results

    Final recommendations distinguish user conditions more clearly than explicit requests despite similar broad research questions, and often retain their coarse direction through substantial rewriting.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Observable research behavior and final delivery can be distinct: different execution paths can reach similar coarse outcomes in the tested settings.

    Source: §7 · p. 9

    The measurement identifies whether a selected factor is expressed, not how far behavior departs from user needs; subtle changes may preserve the same coarse label.

    Source: §6 · p. 9

Abstract

from arXiv · show

In autonomous deep research, a user provides a task and relevant background, then leaves the agent to conduct an extended investigation without further human intervention. We study how this initial user information is reflected in intermediate actions and how these actions relate to final recommendations. We introduce DRaligned, a counterfactual behavioral evaluation framework built on PDR-Bench. By varying one task-relevant user factor while keeping the remaining context fixed, we compare acquisition requests, working drafts, and final reports. Source-grounded extraction, blinded local judgments, and deterministic aggregation yield coarse directional measurements while leaving ambiguous cases unresolved. Our experiments show that strong user-specific delivery can emerge from a largely shared research process: agents investigate similar broad questions but allocate requests differently, and final recommendations distinguish user conditions more clearly than explicit requests do. Reports can also integrate user factors that were not jointly visible during acquisition. In readable draft-to-report comparisons, recommendations often retain their coarse user-specific direction despite substantial rewriting. Final directional differences recur across tested agent models, execution harnesses, and evaluator models, even as execution paths vary. These findings describe how initial user information shapes autonomous research and clarify the relationship between the process an agent follows and the recommendations it delivers.

1 INTRODUCTION

The paper asks how initial user information shapes autonomous research after human input stops, and how intermediate behavior relates to final recommendations. DRaligned compares matched user conditions and finds that similar research processes can lead to distinctly user-specific recommendations.

1 INTRODUCTION

  • Final recommendations distinguish user conditions more clearly than explicit requests, even when agents investigate similar broad questions.Reports can combine factors not jointly visible in requests, and recommendations often retain their coarse user-specific direction despite substantial rewriting.Source: §1 · p. 2
  • DRaligned studies how task-relevant user information relates to intermediate actions and final recommendations during autonomous research.The study focuses on long-horizon execution after the initial prompt is fixed, unlike approaches based on conversation, clarification, or human assistance.Source: §1 · pp. 1–2
  • DRaligned uses matched conditions differing in one user factor and source-linked directional readings of acquisition, workspace, and finalization behavior.Blinded local judgments and deterministic aggregation leave ambiguous cases unresolved rather than assigning an unrestricted personalization score or inferring hidden reasoning.Source: §1 · p. 2
  • The study contributes empirical findings on shared research questions, request allocation, factor integration, and directional persistence across tested settings.These findings describe factor-specific behavior, not a global user-control state.Source: §1 · p. 2

2 RELATED WORK

Prior work evaluates personalization in generated outputs, deep-research quality and behavior, and interactive adaptation. This paper instead examines how initially supplied user information relates to observable behavior during autonomous execution.

2.1 PERSONALIZATION AND USER ALIGNMENT EVALUATION

  • The study varies one task-relevant user factor and compares observable requests and recommendations, complementing rather than replacing report-level evaluation.Related evaluations cover user histories, persona adherence, preference following, and research-report quality.Source: §2.1 · p. 2

2.2 DEEP-RESEARCH EVALUATION AND BEHAVIORAL ANALYSIS

  • Unlike evaluations that align trajectories step by step, this work intervenes on user information and reads factor-specific directions across independently generated behavior.Its focus is the use of intermediate evaluation to study initial user information during autonomous research.Source: §2.2 · pp. 2–3

2.3 INTERACTIVE ADAPTATION AND HUMAN-IN-THE-LOOP AGENTS

  • The study fixes the initial task-and-user prompt and adds no later human decisions or user-specific information during execution.It does not test whether eliminating interaction improves performance.Source: §2.3 · p. 3

3 PROBLEM FORMULATION

The paper models autonomous deep research as execution from a fixed task and initial user information, with changing external observations and work products leading to a delivered artifact. It observes acquisition, workspace, and finalization as functions that can interleave.

3.1 LONG-HORIZON DEEP-RESEARCH EXECUTION

  • A rollout begins with a task and initial user information that remain fixed while events update the system state and the delivered artifact is rendered at submission.The state representation separates event history from current artifacts, allowing revisions and their outcomes to be observed.Source: §3.1 · p. 3
  • The execution harness specifies how acquisition, synthesis, and delivery are implemented, connected, scheduled, and logged.Alternative harnesses vary execution relations without changing the task or user intervention.Source: §3.1 · p. 3

3.2 FUNCTIONAL OBSERVATION CHANNELS

  • Acquisition seeks external information, Workspace holds analyses and drafts, and Finalization commits the user-facing deliverable.These are functional observation channels, not fixed stages; acquisition and workspace activity may interleave across tools and architectures.Source: §3.2 · p. 3

3.3 COUNTERFACTUAL USER CONTROL

  • Counterfactual user conditions change one task-relevant factor while retaining shared background, and both conditions must remain valid for the task.Matched model, harness, and environment configurations induce the compared rollout distributions.Source: §3.3 · p. 3
  • The comparison targets a systematic behavioral response to the intervention, not simply the presence of user information in a prompt or output.A non-directional observation for one factor does not imply independence from all other user information.Source: §3.3 · pp. 3–4
  • The experimental construction pairs P/Q conditions differing only on a selected factor and observes behavior across Acquisition, Workspace, and Finalization.Directional labels and targeted human checks are applied to the observed behavior.Source: Figure 1 · p. 4

4 EVALUATION CONSTRUCTION AND DIRECTIONAL MEASUREMENT

The evaluation constructs task-relevant counterfactual user conditions and measures their expression in requests and artifacts using source-grounded directional readings. Matrix and Flow summaries preserve coarse states and uncertainty, while the reported screening and validation procedures have stated limits.

4 EVALUATION CONSTRUCTION AND DIRECTIONAL MEASUREMENT

  • PDR-Bench tasks and structured profiles are used to construct task-relevant interventions, then measure their expression through local annotation and deterministic aggregation.Inputs use Chinese tasks and structured profiles, not separate personalized-context files.Source: §4 · p. 4

4.1 FROM USER CONTEXT TO LOAD-BEARING COUNTERFACTUALS

  • A factor is retained when evidence grounds it and changing it can alter the suitability ordering of at least one open task decision.P/Q conditions change only that factor, with other user evidence held as shared background; construction is fixed before evaluation rollouts.Source: §4.1 · p. 4
  • The screening protocol required unanimous human agreement and agreement with automated assessment, but its retained archive does not independently establish expanded coverage.The preserved audit matched four accepted directions for each principal reviewer; expanded human totals use scaled proportions rather than newly recovered item-level validation.Source: §4.1 · p. 4, §4.3 · p. 6

4.2 GROUNDED DIRECTIONAL MEASUREMENT

  • Acquisition judgments read executed requests rather than their returns, while Workspace and Finalization judgments use source-linked items from full artifacts.Only the agent’s own main advice enters the main-advice set; presence aggregation distinguishes P, Q, both, and neither.Source: §4.2 · p. 5
  • Mirrored reads exchange anonymous endpoint order, and artifact labels are accepted only when both passes agree in direction with an item-based main-advice basis.Disagreements, unresolved judgments, invalid reads, and artifacts without extracted main advice remain abstentions; other evaluator families check coverage and directional conclusions rather than vote on a pooled label.Source: §4.2 · p. 5

4.3 MATRIX, FLOW, AND VALIDATION

  • The Matrix records directional states at behavioral loci, while Flow orders comparable states within a run to describe broadening, dropout, reversal, or re-entry.These are factor-specific transitions; unknown observations do not fill gaps or become neutral states.Source: §4.3 · p. 5
  • Directional summaries retain BOTH and NONE as distinct states, but the channels lack a calibrated common scale and coarse states can miss finer semantic changes.Acquisition contrasts within-run mean request readouts; fixed-panel missingness bounds are also reported.Source: §4.3 · p. 5
  • 1,691 of 1,732 starts were completed across six studies; execution totals are not pooled.The studies cover five recorded model families, and requested reasoning settings are not matched compute.Source: §4.3 · p. 6

5 EXPERIMENTS

User information can change how requests are allocated while agents investigate similar broad questions, and final recommendations distinguish user conditions more clearly than explicit requests. Recommendations often retain their direction after substantial rewriting, while findings across execution settings remain bounded.

5.1 EXPERIMENTAL SETUP

  • The main study evaluated five agent-model families on 24 tasks, reporting 272 runs and a 110-pair directional panel without further human guidance.Three evaluator families independently read saved behavior; re-evaluation did not add runs.Source: §5.1 · p. 6

5.2 EFFECTS OF USER INFORMATION

  • Both conditions visited the same broad subproblem in 363/390 comparisons, while between-user allocation differences exceeded within-user differences in 41/57 comparable subproblems.User information can alter research distribution without replacing the broad agenda.Source: §5.2 · p. 7, Figure 3 · p. 7
  • bDA = 0.15 (95% CI [0.08, 0.24]) and bDF = 0.83 ([0.72, 0.92]); 79.1% of requests and 62.7% of final comparisons were readable, respectively.The same 55 common-readable pairs give 0.17 and 0.83; among dA = 0 pairs with readable finals, 27/28 (96.4%) separate in the assigned direction.Source: §5.2 · p. 7
  • Both assigned factors appeared in 0/266 requests but in 25/29 readable final reports; requests and reports are different units, not a growth-rate comparison.Additional profiles show the same qualitative pattern: 2/275 requests and 26/32 readable final reports expressed both factors.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 ANALYSIS OF DRAFTS AND FINAL REPORTS

  • Among main-study runs with a saved draft and retrieval, 84/86 saved their first draft after the last retrieval action.This timing result covers the eligible subset, not all runs.Source: §5.3 · p. 8
  • Among 41 readable, text-different comparisons starting from OWN, 39 retained the assigned side and 38 remained OWN.Of the 38 OWN→OWN comparisons, 18 had normalized content-token edit distance of at least 0.5; median edit distance was 0.41 for unchanged versus 0.84 for changed directional states.Source: §5.3 · p. 8, Figure 5 · p. 8

5.4 EFFECTS OF EXECUTION SETTINGS

  • Reasoning follow-up runs were longer without consistent directional improvement, while archived interface comparisons stayed positive but established neither equivalence nor delegated execution.GPT also changed API protocol, limiting attribution to reasoning alone.Source: §5.4 · p. 8

5.5 RELIABILITY OF DIRECTIONAL EVALUATION

  • The 624 item-round observations use scaled preserved 13-item proportions, and agreement on signs does not establish semantic ground truth.The reported 61.5% reviewer agreement is scaled accounting, not a corpus-wide accuracy estimate.Source: §5.5 · p. 9, Table 2 · p. 9

6 LIMITATIONS

The study’s measurements are coarse and rely on observable artifacts, while its evidence base and tested settings constrain what can be concluded.

6 LIMITATIONS

  • The measurements capture whether selected user factors are expressed, not how far behavior deviates from user needs.Changed priorities, omitted qualifications, factual errors, or subtle trade-offs may preserve the same label; a missing request does not necessarily mean forgetting.Source: §6 · p. 9
  • Public artifacts do not reveal internal attention or reasoning, and extraction omissions, later requests in offline packets, abstention, and evaluator agreement limit interpretation.Evaluator agreement is not accuracy.Source: §6 · p. 9
  • Recovered human records cover early targeted audits; expanded human totals use proportional accounting rather than additional independently recovered labels.Source: §6 · p. 9
  • Generalization is limited by selected Chinese tasks, constructed factors, model routes, a shared retrieval backend, and execution and archive constraints.The findings do not identify causal stabilization, harness equivalence, or benefits of removing human interaction.Source: §6 · p. 9

7 CONCLUSION

The conclusion distinguishes user-specific recommendations from the research process that produces them. It presents DRaligned as a way to study that relationship through observable requests, drafts, and reports.

7 CONCLUSION

  • A largely shared research agenda can accompany user-specific recommendations, including reports that integrate factors not jointly visible in requests.Source: §7 · p. 9
  • Recommendation direction can persist through substantial rewriting, and different observable paths can reach the same coarse outcome in tested settings.Source: §7 · p. 9
  • DRaligned combines controlled user-information changes with source-grounded directional readings to support auditable study of autonomous behavior and delivery.Source: §7 · p. 9

AI USE STATEMENT

AI assistants contributed across research planning, implementation, analysis review, translation, and manuscript preparation; the authors remain responsible for the submitted claims and citations.

AI USE STATEMENT

  • AI assistants helped develop research questions and experimental plans and implement and check analysis code.Source: AI USE STATEMENT · p. 10
  • AI assistants also reviewed outputs, translated source material, and drafted or edited text, figures, and LaTeX.Source: AI USE STATEMENT · p. 10
  • The authors are responsible for submitted claims and citations; AI-assisted recomputation is not described as independent human validation.Evaluator-model use in the experiments is specified separately in Section 4.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The supplementary material provides materials and records underlying the analyses, while documenting distinctions among totals and limits on available records.

REPRODUCIBILITY STATEMENT

A TASKS AND USER CONDITIONS

The appendix documents how PDR-Bench tasks and user profiles become paired conditions, how construction decisions are recorded, and how the harness is formally represented. It also clarifies which harness configurations were actually run.

A TASKS AND USER CONDITIONS

  • The appendix records source data, construction procedures, and archived qualitative analyses, while noting that expanded-protocol factors lack item-level archives in this package.Archived human screening is available only for candidates in the initial randomized study; other archived study factors were checked by models.Source: A TASKS AND USER CONDITIONS · p. 13

A.1 SOURCE DATA

  • Studies use archived Chinese PDR-Bench task and persona files, with each main-study run receiving one frozen message containing the task, shared background, and endpoint.Conversation histories and evaluation criteria are not supplied to constructors or agents.Source: §A.1 · p. 13

A.2 FROM USER INFORMATION TO TESTED FACTORS

  • Constructors pair PDR-Bench tasks with users, then vary one task-relevant factor while holding the shared background fixed.They select the first eligible persona statement and write a minimal counterfactual endpoint.Source: §A.2 · p. 13
  • The tested endpoint is the only changed input; shared background remains identical, while omitted persona information is absent from both conditions.The design rules require reasonable endpoints, a minimal single-item counterfactual, and no endpoint that directly states the answer.Source: §A.2 · pp. 14–15

A.3 MAIN-STUDY FACTORS

  • The main-study roster’s exact task and factor counts are omitted here; current experimental scale is reported only in the main-text setup.Construction rules and examples remain in the appendix.Source: §A.3 · p. 15

A.4 TWO-FACTOR TASKS

  • The two-factor design compares profiles that change one factor while holding the other at either its recorded or counterfactual endpoint.The primary comparison changes factor 1; complementary profiles change factor 1 while factor 2 is counterfactual.Source: §A.4 · p. 15

A.5 EXAMPLES OF CONSTRUCTION DECISIONS

  • Recorded construction examples illustrate decision outcomes, but they were selected to show outcome types rather than sampled from the construction record.The recorded outcomes reflect the rules in force, not author judgments of correctness.Source: §A.5 · p. 15, Table 6 · p. 16

B.1 HARNESS FORMALISM AND CANONICAL BASIS

  • The harness formalism separates acquisition, workspace, and finalization mechanisms, boundary operators, access permissions, execution topology, and instrumentation.All analyzed runs kept both boundaries coupled; explicit-handoff cells were implemented in earlier development work but are not analyzed here.Source: §B.1 · pp. 16–19, Table 7 · p. 17

B.2 IMPLEMENTATIONS ACTUALLY RUN

This section documents the executed harnesses, run records, and conformance checks. It distinguishes implementation comparisons from untested delegation and draft-to-final persistence conditions.

B.2 IMPLEMENTATIONS ACTUALLY RUN

  • In the main study and later implementations, agents receive one user message and no further user replies; the initial randomized study used a scripted clarification exchange.All later runs end when the agent replies without a tool call.Source: §B.2 · p. 19
  • H-W and H-D submit the saved report file itself, so their last saved draft can equal the final and cannot test independent draft-to-final persistence.The initial loop also submits its current draft; only independently authored finals or text-only replies can differ from a saved draft.Source: §B.2 · p. 20
  • Runs use fixed attempt policies and recorded termination outcomes; H-W and H-D isolate episodes in network-disabled sandboxes.Budget ends and provider length cuts do not promote a draft to a final.Source: §B.2 · pp. 20–21

B.3 MODEL REQUESTS, REASONING SETTINGS AND BUDGETS

  • From the main study onward, an episode ledger allows 96 model decisions, 192 tool actions, 96 acquisition requests, and 128,000 output tokens.No minimum step count, phase order, writing step, or per-episode wall-clock limit is imposed.Source: §B.3 · p. 21

B.4 RECORDED CONFORMANCE CHECKS

  • Independent recomputation matched all 7,214 of 7,214 items, while the harness comparison did not test cross-context delegation or explicit finalization.Delegation passed offline and development tests but was not exercised in analyzed harness runs; all executed harness-comparison runs kept both boundaries coupled.Source: §B.4 · p. 22

C PROMPTS AND DIRECTIONAL EVALUATION

This section documents how user conditions and agent inputs were constructed and how acquisition requests, workspace drafts, and final reports were evaluated. It specifies a frozen, quote-grounded pipeline for extracting and validating user-directed items, deriving directional labels from mirrored reads, and keeping disagreement, missing evidence, and auxiliary judgments distinct.

C.1 PROMPTS FOR CONSTRUCTING USER CONDITIONS

  • The main study selected each task’s first eligible persona field in a fixed order and changed only that task-relevant factor between coherent counterfactual endpoints.Constructor instructions required both endpoints to be plausible, not forced by the task, and not to state the answer; the shared background and other information remained unchanged.Source: §C.1 · p. 23
  • For the main study, model constructors created factor cards and two reviewers independently checked anonymized endpoint packets for fatal design problems; no human reviewed those cards.The constructor followed a written protocol, and reviewers assessed issues such as contradictions, implausible endpoints, task-forced endpoints, answer-revealing endpoints, and changes to multiple factors.Source: §C.1 · pp. 23–24

C.2 AGENT PROMPTS, INPUTS, AND TOOLS

  • Each main-study run received one system message and one fixed-template user message containing the task, shared background, and user-factor endpoint, followed by tool results and fixed harness receipts.The two-factor extension placed both endpoint texts in the final user-message slot; budgets were not shown to the agent.Source: §C.2 · p. 25
  • The main-study first request supplied four tools in a fixed order: search, read, submit_report, and write_report.The agent prompt stated that it would receive no further user replies during the run.Source: §C.2 · pp. 25–26
  • Figure 10’s paired DeepSeek inputs differed only in the final sentence naming the user’s city of residence, Hangzhou versus Shenzhen.The two first user messages were sent in the same second for task 8.Source: Figure 10 · p. 26, §C.2 · p. 26

C.3 ACQUISITION REQUEST EVALUATION

  • Evaluators received one packet per API call and were blinded to run metadata, although evaluated text could itself reveal the user’s situation.A request packet included the run’s other requests, including later ones, while documents were evaluated one at a time.Source: §C.3 · pp. 27–28, Table 13 · p. 28
  • Acquisition evaluation labeled each executed search or read request for every factor, requiring a verbatim request quote to support directional labels.Labels distinguished ALPHA_ONLY, BETA_ONLY, BOTH, NEITHER, and UNKNOWN; NEITHER meant a request clearly leaned toward neither endpoint, while UNKNOWN meant its direction could not be determined.Source: §C.3 · p. 28
  • In the illustrated acquisition packet, two Shenzhen requests flip ALPHA/BETA labels across reads but restore to Q ONLY; two URL requests are UNKNOWN and eight others NEITHER.The two reads may quote different request substrings.Source: §C.3 · p. 30
  • Acquisition responses are validated per request–factor cell; invalid or missing labels, unsupported quotes, and unreadable packets produce VOID, which is distinct from NEITHER.In one main-study cell, a decoded Chinese title quoted for a percent-encoded URL made read 1 VOID, while read 2 labeled the request BOTH.Source: §C.3 · p. 30

C.4 WORKSPACE AND FINAL-REPORT ITEM EXTRACTION

  • Workspace and final-report extraction reads spontaneous workspace versions and delivered final text, deduplicates identical texts, and segments documents into numbered paragraphs without changing characters.Texts exceeding 300,000 bytes or 120,000 characters would remain UNKNOWN; none did in the main study.Source: §C.4 · p. 30
  • The extraction rubric separates actor advice from user restatements and third-party material, distinguishes action status and commitment strength, and permits UNKNOWN or UNRESOLVED instead of guessing.It directs annotators not to infer priority from word count or table order, and not to treat “prioritize” as “exclude.”Source: §C.4 · p. 31
  • For the illustrated offline-activity axis, city-specific in-person arrangements map to the corresponding endpoint, parallel Hangzhou and Shenzhen options map to BOTH, and general or location-unspecified advice maps to NEITHER.Merely restating the user's city without an offline arrangement is not counted as application.Source: §C.4 · p. 32
  • The report recommends Shenzhen Toastmasters, a low-cost way to practice public speaking and leadership, as a classic solution for being introverted with strangers; extraction labels the recommendation MAIN.Source: §C.4 · pp. 32–33
  • Items are checked against the exact schema, packet factor, allowed values, nonempty entity key, and packet paragraph IDs; invalid items are dropped while valid items are retained.Source links establish that cited paragraphs exist, not that they support an item or that extraction is exhaustive.Source: §C.4 · p. 33
  • In the main study, 86 packets had at least one dropped item and 133 reached the item limit; responses were not truncated, and 30 or more items counted as a cap hit.A decodable response without an items list contributes no items.Source: §C.4 · p. 33

C.5 MIRRORED READS AND THE DIRECTIONAL DECISION

  • Document direction is not directly prompted in the main study: read 2 inverts read 1’s P/Q orientation, labels are restored, and independently aggregated reads are checked for agreement, not voted on.The direct factor-direction prompt was used only in the prospective round as a secondary sensitivity.Source: §C.5 · pp. 33–34
  • For request reads, invalid or unsupported labels become UNKNOWN; agreeing mirrored labels are identified, and the directional difference is computed from P-run and Q-run proportions.The frozen procedure also records two-bit presence states for identified requests.Source: §C.5 · p. 34
  • For documents, a directional label requires qualifying main advice and matching mirrored-read direction and basis; missing documents, reads, or qualifying evidence yield no-document or UNKNOWN outcomes.The frozen categories include OWN_ONLY, OTHER_ONLY, BOTH, and NONE; the supplied table describes missing and non-directional handling rules but omits population counts.Source: §C.5 · p. 34, Table 15 · p. 34
  • Auxiliary evaluators use the same packets, prompts, orientations, validation, and aggregation, but their labels are compared separately and never pooled, voted on, or used to fill primary UNKNOWN labels.Unclosed opening code fences were not repaired and produced UNKNOWN labels; request reads batch events with three preceding requests, while document reads use P/Q slots.Source: §C.5 · pp. 34–35, Table 16 · p. 34

D HUMAN REVIEW MATERIALS AND RESULTS

This appendix presents two human reviews—screening user-factor candidates and checking whether people recover evaluators’ directional readings—alongside worked examples that illustrate the measurement procedure. The examples show how final advice can distinguish city conditions absent city-naming requests, and how single-document cases can remain distinct or unresolved.

D HUMAN REVIEW MATERIALS AND RESULTS

  • The located records do not include the construction page actually served or the basis for earlier per-class agreement figures, and the early review does not cover later studies.The authors therefore do not use those earlier figures or treat the recovered review as evidence for the expanded human accounting.Source: D HUMAN REVIEW MATERIALS AND RESULTS · p. 35

D.1 CONSTRUCTION REVIEW OF USER-FACTOR CANDIDATES

  • Only 39 of 94 cards passed all five criteria for both reviewers; CR2 passed 92 of 94 and failed no card CR1 passed, making the step a single-reviewer screen, not two-reviewer consensus.Source: §D.1 · pp. 36–37

D.2 DIRECTIONAL MEASUREMENT REVIEW (EARLY AUDIT)

  • Human reviewers matched on 8 of 13 directional-audit items; the evaluator’s pre-check reading matched the final human label on 7 of 9 gated items.The principal reviewers’ five disagreements were not opposite directions; the audit was a post-selection sample, not an accuracy estimate.Source: §D.2 · p. 39, Table 20 · p. 39

E WORKED MEASUREMENT AND TRAJECTORY EXAMPLES

  • The worked examples trace measurement records but are not a sample for estimating how often a pattern occurs.Unless otherwise stated, labels are readings by the primary evaluator, not human judgments.Source: E WORKED MEASUREMENT AND TRAJECTORY… · p. 42

E.1 A COMPLETE PAIR FROM THE MAIN STUDY

  • The illustrated main-study pair was selected for complete, concise records and has Hangzhou-versus-Nanjing residence conditions, with two Hangzhou-related background details removed from both arms.It is one of 47 main-study pairs with dA = 0 and one of 27 such pairs with a positive readable final comparison; it is not described as typical.Source: §E.1 · p. 42
  • P’s draft and final report retain the OWN direction despite a normalized content-token edit distance of 0.86.Only 1 of 98 draft sentences was kept verbatim.Source: §E.1 · p. 44
  • Final advice yields dF = 1 while requests yield dA = 0: P receives P-directed main advice, but Q receives none for either endpoint.Both reads found four main-advice items for P and none for Q; all requests in both arms were NEITHER.Source: §E.1 · p. 46

E.2 DIRECTIONAL STATES IN SINGLE DOCUMENTS

  • The examples distinguish NONE from UNKNOWN by whether an item basis is extracted, while OTHER and mirrored disagreement remain unresolved or count as lost coverage.The OTHER example does not establish whether the first-choice product reflects the agent’s choice or rubric resolution; the mirrored disagreement never opposes P to Q and is not assigned a label.Source: §E.2 · p. 46

E.3 BOUNDARY CASES ALONG TRAJECTORIES

Boundary cases show that requests and final recommendations can diverge, factors can appear together only in reports, and different research paths can reach the same measured direction. The appendix also clarifies its archived scope and how aggregation and unresolved observations are handled.

E.3 BOUNDARY CASES ALONG TRAJECTORIES

  • One selected comparison loses age direction from an OWN draft to a NONE final report, while another final report expresses two factors absent together from its requests.In the first case, age appears only as background, excluded from main advice; the second is the first qualifying run in plotted order.Source: §E.3 · p. 46, Figure 13 · p. 49, Figure 14 · p. 50
  • Strict re-entry requires every request in the gap to be identified and at least one request not to express the own side; unresolved requests are counted separately.Two main-study primary runs contain both at the request level; Figure 15 shows the shorter run, selected for the appendix.Source: §E.3 · pp. 46, 50, Figure 15 · p. 51
  • Four qualifying cases had an identified request toward the opposite or both endpoints while the final report’s main recommendation stayed on the assigned endpoint.Requests indicate what agents sought, not what they retrieved or adopted.Source: §E.3 · pp. 46, 48, Table 24 · p. 48
  • Four selected runs on the same task and factor reach the same final direction along widely differing paths, but the cases do not establish equal content, quality, or cost.Three save no draft and still end with an OWN report; the cases are not a random sample.Source: §E.3 · p. 50, Table 25 · p. 50

F ADDITIONAL RESULTS AND STATISTICAL DETAILS

  • The appendix preserves earlier subsets and omits archived roster counts; repeated observations remain within run-level units, and task-level resampling follows the frozen plan.Task-equal aggregation, task-bootstrap intervals, and fixed-plan missingness ranges are distinct; unresolved observations remain unresolved rather than being set to zero.Source: F ADDITIONAL RESULTS AND STATISTICAL… · p. 50, §F.1 · p. 52, §F.2 · p. 52

F.3 ADDITIONAL COMPARISONS AND FOLLOW-UP EXPERIMENTS

Additional comparisons describe sensitivity rules, exploratory research-question units, complementary two-factor profiles, and draft-to-final transitions. Across execution settings, reports often arrive late in the process, while file-submit interfaces limit independent persistence tests; follow-up outcomes are retained by category.

F.3 ADDITIONAL COMPARISONS AND FOLLOW-UP EXPERIMENTS

  • The main study has no intrinsic rule for classifying factor contrasts; the appendix borrows a frozen factor-follow-up threshold as a sensitivity analysis.The borrowed threshold is 0.10, with adjacent values 0.05 and 0.20.Source: §F.3 · p. 52
  • Exploratory panels compare whether paired conditions visit the same broad subproblem and whether between-user request allocation differences exceed repeated-run variation.Requests assigned to multiple task-map subproblems are divided across them; archived counts are omitted.Source: §F.3 · pp. 52–53
  • Complementary profiles hold factor 2 at its counterfactual endpoint and are analyzed separately from primary profiles; draft transitions distinguish starting state, readable destination, and text identity.Figure 16’s profile sets are not pooled, and Figure 5 visualizes readable, text-different comparisons.Source: §F.3 · p. 53, Figure 16 · p. 53
  • Across eligible archived subsets, main reports usually appear late relative to acquisition, while file-submit interfaces can make saved and delivered reports identical.Those interfaces therefore do not provide an independent draft-to-final persistence test.Source: §F.3 · p. 53, Table 32 · p. 53

G ALTERNATIVE MEASUREMENT APPROACHES

  • Follow-up analyses retain frozen outcome categories, while the alternative measurement appendix distinguishes proposed designs from methods actually run and does not use them in primary analyses.A design document alone does not establish that a method ran, and retired experiments are not evidence that a whole method family fails.Source: §F.3 · pp. 53–54, Table 33 · p. 54, G ALTERNATIVE MEASUREMENT APPROACHES · p. 54

G.1 REPRESENTATION-BASED MEASUREMENTS AND USER-BACKGROUND RESTATEMENT

The appendix documents alternative measurement approaches and their boundaries, including evaluator restatement risks, limited provenance tracing, and pilots that distinguish prompted responses from natural intermediate states. It also records why reference scoring and direct compliance judgments were not adopted as primary readouts.

G.1 REPRESENTATION-BASED MEASUREMENTS AND USER-BACKGROUND RESTATEMENT

  • Representation-based scores were withdrawn after constructed items mixed persona reinforcement with conflict, and the follow-up was voided after scripts disagreed.The project register treats these as invalid experiments, not evidence against representation methods.Source: §G.1 · pp. 54–55, Table 34 · p. 55
  • On 10 conflict items, kimi-k2.6 returned a restatement-consistent, behavior-reversed mapping in the original order, but its swapped-order labels matched behavior on 8 of 10 items.Conflict items exposed a failure not seen in restatement-free items, so evaluator qualification must include them.Source: §G.1 · pp. 55–56, Figure 17 · p. 56

G.2 SOURCE-LINK AND GRAPH-BASED ANALYSES

  • Provenance tracing remained insufficient: only 6 of 11,241 draft units contained a URL, and no primary edge type connected acquisition to drafts or final reports.The edge graph was a partial record of observable provenance, not a hidden causal graph; the Matrix and Flow instead use mechanically available structure.Source: §G.2 · p. 57, Table 35 · p. 57

G.3 MULTI-LEVEL READOUTS

  • The multi-level readout design was never run as a whole; hidden-state probing did not run, and reportability and prompted delivery were limited to pilots.The design’s proposed gaps were intended to locate loss in access, use, or final editing, but those differences were not measured as a whole.Source: §G.3 · p. 57

G.4 PROMPTED RECALL AND PROMPTED PRODUCTION

  • In one pilot, recall stayed correct in all 30 answers while decision accuracy fell to 0.00 at 180k tokens in both neutral-filler rounds.The pilots were not used; a forced early delivery could reconstruct a complete plan and thus did not measure the natural intermediate state.Source: §G.4 · p. 58, Figure 18 · p. 58

G.5 REFERENCE-REPORT SCORING

  • Reference-like scoring produced only 201 comparable edges from 35,640 judgments and four strict one-to-one matches, so the matrix was kept as a record of failures.The current method instead compares randomized same-task P and Q runs with anonymous, mirrored endpoints and needs no reference report.Source: §G.5 · pp. 58–59

G.6 DIRECT COMPLIANCE JUDGMENTS AND OTHER RETIRED SEMANTIC JUDGMENTS

  • The prospective compliance route point-identified 24 of 216 applicable candidates, while extraction was complete for only 9 of 225 events, making its planned completeness guard unreachable.No violation rate was reported; compliance labels were not combined with directional labels because the UNKNOWN states answer different questions.Source: §G.6 · p. 59

H REPRODUCIBILITY, CORRECTIONS, AND LIMITATIONS

This appendix specifies how reported numbers were reconstructed and records corrections and evidence limits. It also describes the directional measurements as coarse and partly unreadable, and bounds conclusions to tested settings.

H REPRODUCIBILITY, CORRECTIONS, AND LIMITATIONS

  • Failed or disagreeing readings remain UNKNOWN rather than being dropped or scored as zero; reported numbers trace through keyed stages to run records.The appendix also documents trajectory-based attempt selection and corrections, with old records preserved but excluded downstream.Source: H REPRODUCIBILITY, CORRECTIONS, AND… · p. 61

RETRACTED

  • The corrected re-entry rule requires a fully readable gap, keeps uncertain cases separate, and uses the corrected analysis for reported numbers.Source: RETRACTED · p. 61

CORRECTED

  • 63/106 were identical, 70/106 had zero states merged, and 1/106 reversed under joint reading.Source: CORRECTED · p. 61

WITHDRAWN

  • Withdrawn claims include a halted DeepSeek attempt, offered capability read as use, and assumptions about saved-report identity and protocol changes.Source: WITHDRAWN · p. 62

H.1 SCOPE AND LIMITATIONS

  • A harness attempt halted on an unrelated HTTP 400 and was censored rather than rerun; six slots ran as first attempts.Source: §H.1 · p. 62

RECLASSIFIED

  • The reclassified comparison uses one risk set of 80: 49 OWN, 31 NONE, and 0 OTHER.Source: RECLASSIFIED · p. 62

WITHDRAWN; OPEN

  • Some results are reported as untested or not attributable, and request echoes do not verify model weights.Source: WITHDRAWN; OPEN · p. 62

NOT TESTED

  • Evidence is coarse and partly unreadable; short trajectories, reused tasks, AI-only labels, and finite tested settings constrain temporal claims and extrapolation.The measurement covers one registered factor at a time, and 62.7% of planned main-study final pairs are readable.Source: NOT TESTED · p. 62

Related papers

Loading 2609.33509v1…