PaperlayerView exact revision on arXiv

Source-linked AI summary

ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions

Chuanyang Jin, Binze Li, Haopeng Xie, Cathy Mengying Fang, Tianjian Li, Shayne Longpre, Hongxiang Gu, Maximillian Chen, Tianmin Shu

arXiv:2605.20087v2cs.CLcs.AI

TL;DR

Users’ thoughts during human–AI conversations remain largely unexplored. ThoughtTrace pairs real-world multi-turn conversations with self-reported thoughts and shows that thoughts differ from messages and support downstream prediction and alignment.

  • Problem

    Users’ thoughts during human–AI conversations remain largely unexplored, despite encompassing motivations, constraints, expectations, and reactions not expressed in messages.

    Source: §1 · p. 3

  • Method

    ThoughtTrace collects real-world multi-turn conversations with users’ self-reported reasons for sending messages and reactions to assistant responses.

    Source: §1 · p. 3, §3.1 · p. 5

  • Results

    ThoughtTrace shows thoughts differ from messages, resist inference by frontier LLMs, vary in content and conversation stages, and improve behavior prediction (+41.7% relative gain) and alignment (+25.6% win rate).

    Source: §1 · p. 3

  • Takeaways & Limitations

    User thoughts are a distinct data modality for studying cognitive dynamics in human–AI interaction.

    Source: §6 · p. 14

    Externalizing thoughts during collection may shape the interaction by sharpening or polarizing users’ reasoning.

    Source: §6 · p. 14

Abstract

from arXiv · show

Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long-horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, diverse in content, and tied to conversation stages. We further demonstrate the utility of thoughts for downstream modeling. First, thoughts improve user-behavior prediction as inference-time context. Second, thought-guided rewrites provide fine-grained alignment signals for training personalized assistants. Together, ThoughtTrace establishes user thoughts as a new data modality for studying the cognitive dynamics behind human--AI interaction and provides a foundation for building assistants that better understand and adapt to users' latent goals, preferences, and needs.

1. Introduction

ThoughtTrace pairs real-world human–AI conversations with users’ self-reported thoughts, addressing the gap between observable messages and the cognition behind them. The paper characterizes these thoughts and demonstrates their utility for predicting user behavior and aligning language models.

Motivation

  • User thoughts capture motivations, goals, context, constraints, response expectations, and reactions that messages may leave unstated.The introduction frames thoughts as the hidden cognitive context behind messages and argues that capturing them helps bridge observable utterances and hidden user intent.Source: §1 · p. 3

Dataset

  • ThoughtTrace contains 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models.It pairs multi-turn dialogue with thoughts aligned to user and assistant messages, post-task descriptions, and demographic information.Source: §1 · p. 3

Thought utility

  • Thoughts supported user-behavior prediction with +41.7% relative gain and provided fine-grained alignment signals with +25.6% win rate.The paper reports these results as evidence of thought utility for inference-time prediction and model alignment.Source: §1 · p. 3

Analysis

  • ThoughtTrace characterizes thoughts as distinct from messages, difficult for frontier LLMs to infer, diverse in content, and tied to conversation stages.These findings characterize both the conversational and cognitive structure of the dataset.Source: §1 · pp. 3–4

Research directions

  • ThoughtTrace supports studying thought dynamics across users, training models with thought-based supervision, and evaluating thought prediction and user satisfaction.The paper presents these directions as ways to better capture users’ latent goals, expectations, and reactions beyond surface-level utterances.Source: §1 · p. 3

2. Related Work

Prior work has studied real-world human–AI conversations, latent user thoughts, and user simulation, but lacks large-scale real-user thought traces across multi-turn interactions. ThoughtTrace addresses this gap with self-reported thoughts paired with real conversations.

Real-World Human-AI Conversations

User Thoughts

  • Most machine Theory of Mind work targets structured variables grounded in simple contexts, leaving dynamic thoughts during interaction and their systematic large-scale collection underexplored (Baker et al., 2009, 2017).Recent work explores dynamic mental-state inference for assistance (Zhang et al., 2026, 2025; Zhou et al., 2025), while ThoughtTrace proposes collecting and analyzing latent thoughts across multi-turn conversations.Source: §2 · p. 4

User Simulations

  • User simulators rely on prompted or fine-tuned LLMs, have been found biased and unfaithful, and HumanLM still trains on synthetic thoughts (Wu et al., 2026).ThoughtTrace’s real-user thought traces may support more realistic simulator training.Source: §2 · p. 4

3. Data Collection

ThoughtTrace captures users’ unspoken thoughts alongside real-world human–AI conversations. It collects both reasons for user messages and reactions to assistant responses across varied tasks and models.

3.1. What are Thoughts?

  • Thoughts capture users’ unspoken mental content beyond often-lossy utterances, and are annotated as reasons for user messages or reactions to assistant responses.Reasons reveal what users want the model to understand about their needs and preferences; reactions indicate how they perceive the model.Source: §3.1 · p. 5

3.2. Methodology

  • Participants completed two self-defined 10-minute tasks after a tutorial quiz, privately annotating message reasons and assistant-response reactions that were hidden from the AI.Participants could start new conversations or end tasks freely, and attach multiple thoughts to a message.Source: §3.2 · p. 5

3.3. Models Used

  • Participants each interacted with one of 20 models, including frontier and smaller open-weight systems, without knowing which model they were using.The models included GPT-5.4, Gemini 3.1 Pro Preview, Grok 4.20, and Opus 4.6.Source: §3.3 · p. 5

3.4. Data Format

  • Each conversation record stores model and task metadata plus survey responses, while its ordered messages contain content and timestamped thoughts labeled as reasons or reactions.Thought labels draw from seven reason types and five reaction types.Source: §3.4 · p. 6

4. Data Properties

ThoughtTrace combines broad, long-horizon conversations with explicit thoughts that capture information not stated in messages and vary with conversational progress. Its data properties show both the scope of real-world interactions and the distinct, diverse, context-dependent nature of users’ reasons and reactions.

Conversation Property 1: ThoughtTrace captures a representative spectrum of users.

  • Participants span varied backgrounds, ages 18 to 65+, education levels, and occupations, but skew toward ages 18–34 and undergraduate degrees or higher.Most report using AI at least daily, commonly for learning and work, followed by brainstorming, research, and coding; the sample broadly matches frequent generative AI users.Source: Conversation Property 1: ThoughtTrace… · pp. 6–7

Conversation Property 2: ThoughtTrace features long-horizon diverse conversations.

  • ThoughtTrace conversations have a median of 8 turns versus 2 in both WildChat and LMSYS-Chat-1M, spanning seven broad topics and 36 subtopics without one category dominating.The turn distribution peaks around 6–8 turns; Culture & Lifestyle is the most prevalent broad topic, while nine fine-grained subtopics each exceed 5% of the dataset.Source: Conversation Property 2: ThoughtTrace… · p. 7, Figure 3 · p. 7

Conversation Property 3: ThoughtTrace conversations are dominated by task extension.

  • 57.0% of user turns extend, deepen, or build on a prior task, and this pattern grows more prevalent as conversations progress.First requests account for 25.2%, completely new requests 12.5%, re-attempts or revisions 2.9%, and new variations 2.3%.Source: Conversation Property 3: ThoughtTrace… · p. 8, Figure 4 · p. 8

Thought Property 1: Thoughts are different from messages.

  • Message–thought semantic coverage averaged 3.22 for reasons and 2.00 for reactions, indicating substantial latent information absent from users’ messages.Embedding comparisons also show larger semantic shifts for message–reason pairs than for consecutive user messages, with the widest dispersion for reaction–next-message pairs.Source: Thought Property 1: Thoughts are… · pp. 8–9, Figure 5 · p. 9

Thought Property 2: Thoughts are difficult for LLMs to infer.

  • Across three frontier models, mean 1-to-5 similarity was 2.93 for inferred reasons and 2.54 for reactions, showing both are difficult to infer from context.The paper attributes this to multiple plausible thoughts fitting the same context and users’ unobservable constraints, stakes, or interpretations.Source: Thought Property 2: Thoughts are… · p. 9

Thought Property 3: Thoughts are diverse in content.

  • ThoughtTrace thoughts span seven reason categories and five reaction categories, including reasons for task motivation (36.9%) and continuation (21.4%), and explicit affirmation reactions (72.2%).Other reasons include Context Grounding & Constraints (13.1%), Content Expectation (11.5%), and Style Expectation (5.0%); reactions include Content Relevance (11.9%), Presentation Style (6.4%), Scope Fit (6.1%), and Partial Satisfaction (3.4%).Source: Thought Property 3: Thoughts are… · pp. 10–11, Figure 6 · p. 10, Figure 7 · p. 10

Thought Property 4: Thought dynamics depend on conversation stages.

  • Thought types shift with conversation stage but show no clear relationship with topic or length: task motivation dominates early turns, while task continuation grows later.Explicit Affirmation rises from 67% in early stages to 79% later, while Presentation Style and Scope Fit reactions decline; users predominantly extend conversations regardless of reaction type.Source: Thought Property 4: Thought dynamics… · p. 11, Figure 8 · p. 11

5. Utility of Thoughts

ThoughtTrace’s user thoughts provide actionable signals beyond conversation transcripts, with experiments showing their utility for predicting user messages and improving model alignment.

5.1. Thoughts Predict User Behavior

  • Prediction adds annotated reasons and reactions at inference; alignment identifies unsatisfactory responses from dissatisfaction labels, revises them using thoughts, then trains on preference pairs.The prediction experiment compares conversation history alone with history augmented by thoughts; the alignment experiment uses DPO training.Source: §5.1 · p. 12, §5.2 · p. 13
  • Access to thought annotations at inference time substantially improves next-message prediction across three frontier models, raising average performance from 21.6 to 30.6, a 41.7% relative gain.Claude Opus 4.6 gained 14.2, while GPT-5.4 and Gemini 3.1 Pro Preview gained 6.0 and 6.8, respectively.Source: Table 1 · p. 12, §5.1 · p. 12
  • Future work could jointly predict thoughts and user messages for user simulation, incorporate reasons alongside reactions in alignment training, and extend thought-guided supervision to reward modeling and online alignment.The current alignment experiments use reactions only.Source: §5.1 · p. 13, §5.2 · p. 14

5.2. Thoughts Improve Model Alignment

  • Thought-guided rewrites gained +4.5% over message-guided rewrites, +25.6% over base Qwen3.5-4B, and +6.6% over the WildChat baseline in style-controlled Arena-Hard performance.The rewrites were used to create preference pairs for DPO training.Source: Table 2 · p. 13, §5.2 · p. 13
  • Thoughts surfaced 1,000 dissatisfaction instances versus 450 in messages (2.2× more), combining ground-truth reactions with revision guidance rather than behavioral proxies.The authors interpret this as denser supervision and as evidence that thoughts capture richer dissatisfaction and revision signals than users explicitly state in messages.Source: §5.2 · pp. 13–14

6. Conclusion

ThoughtTrace pairs real-world human–AI conversations with users’ self-reported thoughts, establishing thoughts as a distinct modality that reveals information beyond messages and supports downstream modeling. Its scope is limited by in-situ collection, the focus on consciously accessible reasoning, Prolific recruitment, and evaluation of only two downstream uses.

Conclusion

  • ThoughtTrace establishes user thoughts as a distinct modality: they capture latent information beyond messages, are difficult for frontier LLMs to infer, and vary in content and across conversation stages.The dataset pairs real-world human–AI conversations with users’ self-reported thoughts.Source: §6 · p. 14
  • Thoughts improve user-behavior prediction at inference time and provide fine-grained alignment signals for training personalized assistants.The paper frames user thoughts as a foundation for studying human–AI cognitive dynamics and for evaluating success beyond surface utterances, including intent, satisfaction, and subjective experience.Source: §6 · p. 14

Limitations and Future Work

  • ThoughtTrace’s limitations include possible interaction effects from thought externalization, unobserved subconscious judgments, Prolific selection effects, and evaluation of only two downstream use cases.The authors note that demographic analysis suggests the sample remains broadly representative of frequent AI users, and leave more comprehensive empirical investigation to future work.Source: §6 · p. 14

Author Contribution Statement

The authors divided work across project conception, data and analysis, experiments, advising, and manuscript preparation, with Chuanyang contributing to several stages.

Project and data

Property analyses

Thought utility experiments

Advising

Manuscript preparation

A. Details of Models Used in ThoughtTrace · B. Additional Results

ThoughtTrace includes 1,058 high-value users, 2,155 timestamped conversations, 17,058 interaction turns, and 10,174 thought annotations collected through a chatbot service powered by 20 language models. The appendix also specifies response-display settings and model-wise dataset statistics.

A. Details of Models Used in ThoughtTrace

  • ThoughtTrace comprises 1,058 high-value users, 2,155 timestamped conversations, 17,058 interaction turns, and 10,174 thought annotations across 20 language models.Source: A. Details of Models Used in… · p. 18
  • All models use an inference temperature of 0.7; for thinking-mode models, users see only the final response, not intermediate reasoning traces.A loading indicator appears during the thinking process.Source: A. Details of Models Used in… · p. 18
  • Table A1 reports model-wise counts of users, conversations, messages, and thought annotations, and indicates whether each model’s weights are publicly available.Source: Table A1 · p. 18

B.1. Qualitative Examples of Frontier Model Failures in Thought Inference

Four frontier-model failure cases show that Reason and Reaction inference can fail when models follow local textual cues instead of users’ underlying intent or affect.

B.1. Qualitative Examples of Frontier Model Failures in Thought Inference

  • All four examples scored 1/5: frontier models used local textual cues and missed the user’s underlying intent or affect.The cases span GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro Preview.Source: §B.1 · p. 19
  • For Reason, GPT-5.4 mapped “it” to switches instead of licensing concerns, while Claude Opus 4.6 missed the user’s intent to personalize an emerging time-management solution.The user was raising another obstacle because they felt close to finding a solution; the model treated it as a standalone advice request.Source: §B.1 · pp. 19–20
  • For Reaction, GPT-5.4 inferred dissatisfaction from a microwave clarification despite appreciation, while Gemini misread frustration at unsolicited, over-scoped advice as satisfaction.The users appreciated the meal answer’s detail and macros, but the gym-plan user was annoyed that the reply added nutrition, cardio, and tips beyond the request.Source: §B.1 · pp. 19, 21, 22

B.2. Qualitative Examples of User Behavior Prediction

Three qualitative examples compare frontier models’ next-message predictions with and without annotated thoughts, using an LLM judge against the actual message. Thought-aware predictions win in two examples but lose in one, showing that accurate thoughts do not always predict what users will say next.

B.2. Qualitative Examples of User Behavior Prediction

  • Thought-aware predictions win in the Paris and anxiety examples, but lose in the piano example, where context-only predictions better match the user’s next message.The LLM judge’s rulings agree with a human reader’s assessment; the failure shows that predictors must identify which thoughts users will surface next.Source: §B.2 · pp. 22, 24
  • In the Paris example, “too much data, narrow it down” shifts all three models toward brief requests for top activities, matching the user’s actual request.Without thoughts, each model assumes the user will answer the assistant’s questions about dates, budget, or interests.Source: §B.2 · pp. 22–23
  • In the anxiety example, the reaction “It always ask me questions” leads GPT-5.4 and Gemini to predict complaints about the assistant’s questioning, matching the user’s frustration.Claude’s predictions are similar with or without thoughts, though the judge marginally favors its thought-aware version in tone.Source: §B.2 · pp. 22, 24
  • In the piano example, thought-aware predictions overemphasize easier songs and formatting, while the user accepts the realistic timeline and redirects advice to classical and jazz.The context-only predictions already capture the genre-focus pivot, despite annotations accurately recording the user’s formatting preference and growing acceptance of the assistant’s realism.Source: §B.2 · pp. 25–26

B.3. Conversation, Message, and Thought Lengths · B.4. Full Topic Distribution · B.5. Task Descriptions and AI Expectations

These appendices show that ThoughtTrace includes longer conversations, brief but evolving thoughts, and diverse topics. Users describe concrete planning and problem-solving tasks while preferring clear, actionable, tailored guidance.

B.3. Conversation, Message, and Thought Lengths

  • User prompts have a median length of 13 tokens versus 561 for assistant responses, whose lengths vary widely and depend only weakly on turn position.Assistant responses fluctuate around 480–810 tokens per turn; their ±1 standard deviation bands span from near zero to well over 2,000 tokens, so small mean differences do not indicate a strong trend.Source: §B.3 · pp. 26–27, Figure A2 · p. 28
  • Nearly 60% of WildChat and over 90% of LMSYS-Chat-1M conversations fall below 1k tokens, while ThoughtTrace spreads more evenly across 2k–5k and retains a tail beyond 10k.This distribution indicates that ThoughtTrace includes longer, more information-dense exchanges.Source: §B.3 · p. 26, Figure A1 · p. 27
  • Thoughts peak at 8–12 tokens (27.1%), about three quarters are 4–20 tokens, and average length settles at 11–13 tokens from turn 4 onward after reaching 15–18 at turns 1–2.Fewer than 3% exceed 40 tokens.Source: §B.3 · p. 27

B.4. Full Topic Distribution

  • Among the most prevalent subtopics are Business & Finance (9.3%), Travel & Tourism (9.0%), Lifestyle (8.9%), Food & Dining (8.4%), Geography (8.0%), and Education (7.7%).Travel & Tourism, Lifestyle, and Food & Dining lead within Culture & Lifestyle, the largest parent category.Source: §B.4 · p. 27
  • Relationships (6.2%), Health & Medicine (5.5%), and Technology & Software (5.3%) also have notable shares, while Politics & Elections (0.3%), News & Current Affairs (0.2%), and Fiction & Fanfic (0.1%) form part of a long tail.The distribution indicates coverage of everyday assistance-seeking rather than concentration in a narrow domain.Source: §B.4 · p. 27

B.5. Task Descriptions and AI Expectations

  • Task summaries emphasize structured, goal-oriented activities such as planning, problem solving, daily routines, meal preparation, and study plans.Users commonly frame requests around organization, decision-making, productivity, and iterative real-world problems.Source: §B.5 · p. 28
  • Task expectations emphasize accessible, actionable guidance—especially step-by-step information, ideas, and advice—with clarity, specificity, detail, recommendations, and context-aware tailoring also salient.Users evaluate responses by whether they translate into executable steps.Source: §B.5 · p. 28

B.6. Embedding Differences Between Messages and Thoughts

Embedding analyses show consecutive messages remain similar, message–reason pairs retain shared semantic grounding but are distinguishable, and reactions differ most from subsequent messages.

  • Reaction–next-message pairs had the largest displacement magnitudes and widest dispersion (25th/median/75th percentile distances: 3.93/6.62/9.75), versus current–next messages (median 1.96, 75th percentile 6.89) and message–reason pairs (0.77/3.71/6.94).Current–next points clustered near the origin with comparatively limited spread and approximately isotropic directions. Message–reason and reaction–next-message displacement vectors showed directional organization.Source: §B.6 · p. 29
  • At the group level, current and next messages largely overlap, message and reason embeddings overlap substantially, and reactions and subsequent messages are well-separated.Reasons remain semantically aligned with the original message but are distinguishable at the distribution level; reactions exhibit a larger shift relative to subsequent messages.Source: §B.6 · pp. 29–30
  • Centroid/MMD/AUC: current–next messages, 0.120/0.096/0.721; message–reason, 0.225/0.182/0.977; reaction–next-message, 0.320/0.257/0.988.These measures show the smallest separation for consecutive messages and the largest for reactions versus subsequent messages.Source: §B.6 · p. 30

B.7. Relationships Between Thought Types and Conversation Properties … C.6. Safeguards

ThoughtTrace’s supplementary analyses relate thought types to conversation dynamics and model reactions, while its data-collection procedures elicit private, in-the-moment thoughts under consent and safeguards. The sections also describe surveys, interface design, and limited data cleaning.

B.7. Relationships Between Thought Types and Conversation Properties

  • Reasons shift from Task Motivation in first requests to continuation-, context-, and expectation-related reasons later; users mostly extend tasks regardless of reaction valence, while thought types are largely topic- and length-independent.Explicit affirmation is associated with slightly shorter remaining conversation length, but the effect is not significant.Source: §B.7 · p. 31, Figure A8 · p. 33

B.8. User Satisfaction Across Different Models

  • Explicit affirmation accounts for 55–82% of reactions across models; Gemma-4-26B-A4B-It and Minimax-M2.7 exceed 80%, while Gpt-Oss-120B is near 55% and has the highest Scope Fit share.Content relevance is the second-largest reaction category across models.Source: §B.8 · p. 32, Figure A13 · p. 38

C.1. User Consent

  • Participants were recruited through Prolific, paid above applicable minimum wage, gave informed consent, and were assured anonymity and the right to withdraw.The study had prior IRB approval; participation was voluntary, and participants self-reported English fluency.Source: §C.1 · p. 33

C.2. Tutorial

  • A plain-language tutorial guided participants through a 10-minute example task and asked them to record reasons for each message and reactions to each AI response.Participants were instructed to annotate where and why they were satisfied or dissatisfied with responses.Source: §C.2 · pp. 33–34

C.3. Chat Interface

  • The web chat interface used a 10-minute timer and separate reason and reaction annotation controls; saved thoughts were private and not sent to the AI.The application was built with HTML, CSS, and JavaScript, with Firebase Firestore for real-time data persistence.Source: §C.3 · p. 36

C.4. Post-Chat Surveys

  • After each task, participants described the task and their expectations of the chatbot; after both tasks, they completed a demographic and AI-use survey.The background survey asked about age, gender, education, occupation, AI-chat usage frequency, and main purposes for using AI.Source: §C.4 · p. 37

C.5. Data Cleaning

  • Data cleaning retained most collected data, removing only cases involving unusually fast completion, chatbot delays that led to low-quality consecutive messages or annotations, or extremely low-quality incomplete submissions.Some affected conversations were removed in part and others in full.Source: §C.5 · pp. 37–38

C.6. Safeguards

  • Safeguards include IRB-approved collection, informed consent, anonymity, coarse demographics, and a research-use CC-BY-4.0 license; the dataset card prohibits re-identification, exploitation, and treating self-reports as complete cognition.The card documents demographic biases and reactivity from thought elicitation and defines the dataset’s validated scope as studying latent thoughts in multi-turn human–AI interaction.Source: §C.6 · p. 38

C.7. Limitations … D.2. Conversation Property 2: ThoughtTrace Features Long-horizon Diverse Conversations

The limitations section treats thought annotations as necessarily reactive, conscious-only, and recruited rather than fully in-the-wild; accompanying analysis details describe how participant demographics, conversation lengths, message lengths, and topics are characterized.

C.7. Limitations

  • Thought collection can reshape interactions, so ThoughtTrace records thoughts-as-reported rather than thoughts-as-occurred, despite interface steps to reduce reactivity.Participants may consciously or unconsciously simplify preferences or intentions to make them easier to report; the interface minimizes interruption and framing cues but cannot eliminate this effect.Source: §C.7 · p. 39
  • ThoughtTrace captures consciously accessible, verbalizable reasoning, not subconscious processes or a complete account of the cognition driving behavior.The paper frames annotations as a conscious overlay on underlying cognition, rather than a transcript of it.Source: §C.7 · p. 39
  • Prolific recruitment introduces a modest selection effect, but demographic analysis suggests the resulting distribution shift is small relative to consented annotations at scale.The authors cite a diverse spectrum of AI users and everyday use cases consistent with frequent AI users in the real world.Source: §C.7 · p. 40

D.1. Conversation Property 1: ThoughtTrace Captures a Representative Spectrum of Users

  • Participant demographics and usage are aggregated from each annotator’s first post-task survey response across six axes, with fixed-category and free-text responses normalized differently.The axes are age, gender, education, occupation, LLM-use frequency, and purposes of use; purposes are grouped into semantic categories using keyword rules.Source: §D.1 · p. 40

D.2. Conversation Property 2: ThoughtTrace Features Long-horizon Diverse Conversations

  • Conversation length comparisons with WildChat use per-conversation token and turn counts, normalized into percentage distributions for side-by-side plots.Tokens are counted with tiktoken for GPT-4o; token totals use 1,000-token bins through 15k, while turn counts use even bins with the x-axis limited to 1–25.Source: §D.2 · p. 40
  • Prompt and response lengths are analyzed by role using token-count histograms and median markers, with mean message lengths also plotted by alternating turn position.The turn-position analysis covers positions 1–20, retaining user means at odd positions and assistant means at even positions.Source: §D.2 · pp. 40–41
  • Conversation topics are assigned with deterministic, multi-label GPT-5.4 classification against a predefined taxonomy, then grouped hierarchically for a treemap of topic frequency.The method labels all clearly present topics rather than forcing one primary topic, validates returned labels against the taxonomy, and manually groups topics into broader categories.Source: §D.2 · p. 41

D.3. Conversation Property 3: ThoughtTrace Conversations are Dominated by Task Extension … D.9. Thought Utility 2: Thoughts Improve Model Alignment

This section examines how thoughts relate to visible dialogue, user-turn relationships, thought categories, and conversation context, including their role in next-message prediction. It also describes procedures for thought-conditioned prediction and for constructing and evaluating thought- versus message-guided rewrites.

D.3. Conversation Property 3: ThoughtTrace Conversations are Dominated by Task Extension

  • Each subsequent user turn is classified by its relationship to the preceding turn as extension, revision, variation, or a completely new request.The labels are attached to individual messages and stored at the conversation level to analyze how users continue, revise, vary, or shift tasks.Source: §D.3 · p. 43

D.4. Thought Property 1: Thoughts Are Different from Messages

  • Semantic coverage scoring compares user utterances with their associated reasons and reactions to assess how much thought content is expressed in messages.The rubric uses a 1–5 scale, with separate coverage assessments for reasons and reactions.Source: §D.4 · p. 44

D.5. Thought Property 2: Thoughts Are Difficult for LLMs to Infer

  • Thoughts are described as distinct from utterances and difficult for frontier models to reconstruct from dialogue context.The analyses separately assess semantic coverage of reasons and reactions and compare three models’ inferred thoughts with human annotations.Source: §D.4 · p. 44, §D.5 · p. 45

D.6. Thought Property 3: Thoughts Are Diverse in Content

  • Thought labels distinguish reasons such as task motivation, continuation, and reorientation, alongside reactions such as affirmation, partial satisfaction, and dissatisfaction.Reason categories also include content and style expectations, constraints, and social interaction; reaction categories include presentation style, scope fit, and content relevance.Source: §D.6 · pp. 47–49

D.7. Thought Property 4: Thought Dynamics Depend on Conversation Stages

  • Thought patterns are analyzed across normalized conversation stages, topics, multi-turn relationships, and conversation length.The analyses use stage-level category distributions and flows, normalized topic and relationship heatmaps, and positional statistics for total and remaining conversation length.Source: §D.7 · pp. 49–50

D.8. Thought Utility 1: Thoughts Predict User Behavior

  • Thought-augmented context consistently outperforms history-only context for next-message prediction across all three predictor models.The comparison keeps annotations rated at least 4 on a 1–5 quality scale and scores predictions for semantic similarity to the actual next user message.Source: §D.8 · p. 50
  • Next-message prediction compares conversation context alone with context containing thought annotations, scoring predicted messages against actual messages on a 0–100 semantic-match rubric.The rubric ranges from no meaningful overlap to a full match in meaning, ignoring wording, length, and style.Source: §D.8 · pp. 50–51

D.9. Thought Utility 2: Thoughts Improve Model Alignment

  • Thought-guided DPO pairs use filtered dissatisfaction thoughts and pre-response context, while message-guided pairs use classified dissatisfied follow-ups to generate proactive rewrites.Both methods pair the revised response as chosen with the original unsatisfactory response as rejected.Source: §D.9 · pp. 51–52
  • The three rewrite datasets contain 1,000 thought-guided ThoughtTrace instances, 450 message-guided instances from the same conversations, and 1,000 message-guided WildChat instances.The smaller ThoughtTrace message-guided set is intended to enable comparison on identical conversations.Source: §D.9 · p. 52
  • All three runs use Qwen3.5-4B initialization and DPO; evaluation uses Arena-Hard with GPT-4o judging and reports raw and style-controlled win rates.Training uses batch size 64, learning rate 1×10−6, up to 20 epochs, and early stopping on a 10% validation split; Arena-Hard has a reported 98.6% correlation with human preference.Source: §D.9 · p. 53

Related papers

Loading 2605.20087v2…