PaperlayerView exact revision on arXiv

Source-linked AI summary

"I Didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration

Eunsu Kim, Jessica R. Mindel, Kyungjin Kim, Sherry Tongshuang Wu

arXiv:2605.21363v2cs.CL

TL;DR

Existing attribution methods focus on final artifacts, leaving the joint shaping of goals during human–AI collaboration difficult to measure. CoTrace decomposes goals into verifiable requirements and traces direct and indirect influences across dialogue. Across real-world logs, simulations, and a user study, it finds that models shape lower-level requirements, interaction design changes their goal-shaping behavior, and goal-level analyses alter users’ perceived contributions.

  • Problem

    Existing attribution methods focus on final artifacts, missing how humans and AI jointly shape evolving goals during collaboration.

  • Method

    CoTrace decomposes explicit goals into granular, verifiable requirements and traces direct contributions and indirect influences across dialogue turns.

  • Results

    Humans account for 75–89% of SHAPER mass while LLMs account for 96–99% of EXECUTOR mass, with model goal-shaping contributions increasing toward specific requirements.

  • Takeaways & Limitations

    COTRACE supports evaluation, design, and reflection by measuring collaborative goal shaping, exposing contribution dynamics, and informing calibration of AI initiative.

  • Takeaways & Limitations

    COTRACE requires interaction traces with user consent, provides analytical estimates rather than ground-truth responsibility measures, and was evaluated in a limited user-study sample.

Abstract

from arXiv · show

As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.

1 Introduction

Existing attribution methods focus on final artifacts, but miss how humans and LLMs jointly shape goals during collaboration. CoTrace addresses this gap by tracing direct and indirect requirement-level contributions across dialogue and supporting evaluation, intervention, and user awareness.

  • Current attribution tools primarily detect AI involvement in final artifacts rather than tracing how collaboration shapes goals.This leaves instructors, reviewers, and users unable to distinguish AI-originated goals from AI-executed instructions.
  • CoTrace decomposes explicit goals into granular, verifiable requirements and traces their lifecycles across dialogue.The framework links requirements to utterances where concrete decisions occur.
  • Direct contributions explicitly create or modify requirements, while indirect contributions influence another party to formulate new requirements.Indirect influence can arise through clarifying questions, draft artifacts, or exposed errors.
  • CoTrace is evaluated as an evaluation suite, design tool, and reflection tool for measuring goal shaping, manipulating model initiative, and exposing contribution dynamics.The studies use real-world collaboration logs, controlled simulations, and an interactive viewer.
  • The framework aims to support more principled attribution where AI-assisted work is evaluated, credited, or regulated.Its intended settings include contexts where authorship, responsibility, and credit matter.

2 COTRACE: Evaluation Framework for Quantifying Agents’ Goal-Level Contributions in Human–AI Collaboration

CoTrace represents collaboration goals as outcomes paired with checkable requirements, then traces how dialogue actions create, revise, delete, and influence those requirements. Its pipeline aggregates these requirement-level analyses into goal-level contribution matrices validated through manual review and participant judgments.

  • Goals as Outcome–Requirement Tuples: CoTrace represents each goal as a desired outcome paired with independently checkable requirements.This representation makes underspecified goal shaping observable at a granular level.
  • Goals as Outcome–Requirement Tuples: The framework organizes goals hierarchically into parent goals for overall objectives and child goals for specific sub-tasks.Models can contribute more to specific child outcomes and requirements than to parent-level direction.
  • Atomic Actions: Atomic actions connect utterances to requirements by representing minimal communicative units such as requesting, constraining, and providing code.Actions also serve as units for requirement iteration.
  • Direct and Indirect Influence: COTRACE distinguishes direct goal shaping from potential indirect influence arising when earlier actions provide context for later requirements.The framework treats goal shaping as cumulative across preceding interaction actions rather than as a single creation event.
  • Pipeline and Validation: The automated pipeline extracts outcomes and actions, assigns action roles, tracks requirement CREATE, REVISE, and DELETE operations, and aggregates scores into goal-level contribution matrices.Its four-stage operationalization uses LLMs as judges, while validation separately evaluates goal extraction, requirement extraction, and influence labeling.

3 Measuring Collaborative Goal Shaping In the Wild

Across 638 collaboration logs, COTRACE shows that humans usually set overall direction while LLMs increasingly shape specific requirements through direct and indirect interaction.

  • Data and Questions: 638 ShareChat logs across four sustained-collaboration task categories are analyzed to measure who shapes goals and how goal shaping emerges.The tasks are Computer Programming, Data Analysis, Writing, and Planning.
  • Who Shapes Goals: Humans account for 75–89% of SHAPER mass, while LLMs account for 96–99% of EXECUTOR mass across all four tasks.The pattern indicates human-led direction alongside model-led execution in the analyzed logs.
  • Who Shapes Goals: LLM goal-shaping contributions increase with specificity: models rarely shape parent outcomes but contribute more to child outcomes and individual requirements.Models therefore contribute more to elaborating subgoals and requirements than to setting overall direction.
  • Task Effects: Models contribute more requirements in technical, closed-ended tasks, eventually surpassing users in Data Analysis, while their contributions remain substantially lower in open-ended tasks (p < .001).Users show the reverse pattern, contributing relatively more in non-technical, open-ended tasks.
  • How Goals Emerge: Indirect goal shaping rises after users introduce initial requirements, with user-created requirements influenced by assistants steadily increasing during interaction.The analysis identifies 11 recurring interaction patterns grouped into underspecified intent, artifact-triggered elaboration, problem-triggered revision, and interactional steering.
  • System Settings: Chat-based collaboration contributes more requirements than autonomous agents: 33.11% vs. 5.33% in Academic Writing, 47.03% vs. 5.56% in Data Analysis, and 37.74% vs. 18.35% in Planning.All comparisons have p < .001; the authors report that greater agent autonomy coincides with less goal-shaping initiative.

4 Inducing and Evaluating Goal Shaping at Inference-Time

Controlled simulations test whether interaction design and prompting can amplify model goal shaping and whether increased shaping improves collaborative outcomes. The interventions substantially increase assistant-originated requirements, but more shaping does not clearly improve output quality.

  • Simulation Design: The simulations compare agentic and chat-based COGYM settings across Writing, Planning, and Data Analysis, measuring goal shaping, requirement satisfaction, and output quality.The study also tests inference-time prompting strategies based on observed indirect-influence patterns.
  • Prompting Interventions: Two interventions—underspecification and interactional steering—increase assistant-originated requirements relative to the 30.65% base rate.The interventions are designed to create more opportunities for the assistant to participate in goal shaping.
  • Prompting Interventions: Assistant requirement contributions rise from 30.65% to 69.64% under underspecification (+39.0pp) and to 51.47% under interactional steering (+20.8pp), both p < .05.User-created requirements decrease under both interventions.
  • Interaction Setting: Assistants directly contribute to 42.9% of requirements in CHAT-COGYM versus 24.5% in AGENTIC-COGYM (p < .001).Requiring communication before each tool call creates opportunities to articulate plans and propose next steps.
  • Outcomes: Assistant-created requirements are satisfied more often than user-created ones, 75.1% vs. 62.7%, but the gap nearly disappears after excluding same-turn execution.Excluding same-turn cases, satisfaction is 61.5% for assistant-created requirements, and requirement count is essentially uncorrelated with normalized output quality (Pearson = −0.002; Spearman = −0.011).

5 Exposing Goal-Level Dynamics to Users

COTRACE exposes goal-level dynamics to collaborators and evaluators, changing perceptions of who contributed to shaping and executing work. Users report noticing previously hidden AI decisions, while evaluators use the tool as evidence without surrendering judgment.

  • User Study: The user study compares participants’ perceptions before and after they inspect their own human–LLM interaction logs with COTRACE.Ten participants complete paired travel-planning sessions and provide within-subject perception measures.
  • Perception Shifts: Participants’ perceived own execution contribution decreases by 1.8 points, while perceived LLM execution contribution increases by 0.5 points after COTRACE exposure.Perceived goal-shaping contributions also change, with absolute shifts of 1.0 point for participants and 1.6 points for the LLM.
  • User Reflection: 9 out of 10 participants say COTRACE helped them notice collaboration aspects they had not previously been aware of.Several specifically report becoming more aware of implicit LLM contributions and reflecting on their prompting behavior.
  • Third-Party Evaluation: Evaluators revise their reliance estimates in three of four evaluator–student cases after using COTRACE.They report that raw-log review is time-consuming or focuses attention on the interaction’s initial portion.
  • Third-Party Evaluation: COTRACE informs rather than replaces evaluator judgment: evaluators identify broadly similar usage patterns but make substantially different grading decisions, including scores of 0 and 100.The tool surfaces interaction-level evidence while leaving final pedagogical or evaluative judgment to the evaluator.

6 Related Work

Prior collaboration-evaluation frameworks generally assume predefined tasks or criteria, while newer work considers progressively specified goals without fully capturing co-evolving human–AI goal shaping.

  • Prior Work: Existing human–AI collaboration frameworks generally evaluate task performance or collaboration quality under predefined tasks, requirements, or criteria.This related-work characterization covers frameworks cited by the paper.
  • Remaining Gap: More recent studies examine goals that become progressively specified through interaction but do not fully capture how humans and AI jointly shape co-evolving goals.The paper positions its goal-level attribution framework against this remaining coverage gap.

7 Conclusion and Implication

COTRACE provides a goal-level attribution framework for tracing how humans and AI jointly shape goals and requirements. Its findings support using goal-level visibility to improve collaboration and inform evaluation, design, and reflection.

  • COTRACE traces how humans and AI jointly shape goals and requirements throughout collaboration.
  • Increasing model goal shaping does not necessarily improve final outcomes, motivating closer alignment between collaboration and task quality.
  • Interaction design and prompting can amplify or suppress goal shaping, making system design relevant to calibrating AI initiative.
  • Goal-level attribution may support self-awareness and third-party evaluation in settings where authorship, responsibility, and credit matter.

Limitations

COTRACE has boundaries concerning data access, interpretation of contribution scores, and user-study generalizability. Its analyses require interaction traces, do not measure responsibility or quality directly, and may not generalize broadly.

  • COTRACE requires interaction traces and is not a detector of AI contribution from final artifacts alone.Its use is most appropriate when logs are available with user consent.
  • Contribution scores are analytical estimates rather than ground-truth measures of responsibility, quality, or acceptable AI use.A higher AI contribution score does not necessarily imply better or worse task performance.
  • Final judgments about responsibility, quality, and acceptable AI use should remain with human users or evaluators.
  • The user study had a limited sample and involved frequent AI users in a familiar task domain, constraining generalizability.Findings may not generalize to less experienced users or unfamiliar domains.
  • Participants’ revised judgments may partly reflect anchoring to interface scores, although qualitative responses also referenced linked goals, requirements, and rationales.

Ethics Statement

The project reports institutional review, participant compensation, and consent procedures. It also notes the use of software tools for plot clarity and an LLM-as-a-Judge for evaluation.

  • Human-subject studies received approval from CMU’s Institutional Review Board under study number STUDY2026 00000006.
  • All participants received appropriate compensation, with recruitment and payment details provided in Appendix F.
  • Consent was obtained from all participants before their involvement.
  • Claude Code and Cursor were used to improve plot clarity based on original versions.
  • Evaluation adopted an LLM-as-a-Judge approach.

B.1 Implementation

COTRACE operationalizes goal-level attribution by representing collaboration outcomes and requirements, decomposing dialogue into actions, and quantifying direct and indirect influence.

  • Operational Definitions: Goals are explicit, observable, actionable targets represented by a desired outcome and independently verifiable requirements.Requirements are modeled as evolving through CREATE, REVISE, and DELETE operations.
  • Outcome and Action Extraction: The pipeline automatically extracts outcomes and atomic actions, assigns dialogue roles, and links actions to outcome threads.Actions are labeled SHAPER, EXECUTOR, or OTHER, while outcomes retain version history across dialogue blocks.
  • Requirement Extraction: Requirements are linked to origin and contributing actions and tracked through CREATE, REVISE, and DELETE operations.This produces a versioned history of how requirements evolve during collaboration.
  • Influence Labeling: Influence labeling distinguishes direct connections from implicit contextual connections and excludes actions without meaningful semantic relations.Direct actions operate on requirements, whereas implicit actions provide context that motivates or triggers them.
  • Quantifying Contribution: Contribution scores aggregate action-level influence into speaker-level and role-level measures across requirements within each outcome.Role-level attribution uses SHAPER and EXECUTOR labels, and influence scores assign maximal direct influence to actions in a requirement’s origin turn.

B.3.1 Manual validation

Manual validation evaluates goal, requirement, and influence extraction separately, while participant validation assesses alignment between analyses and users’ perceptions. Errors arise mainly from confusing requirements with outcomes, misattributing authorship, or overstating influence links.

  • Over 100 entities from 37 dialogues were manually validated for each extraction step.The validation covers goals, requirements, and influence labels separately.
  • 66.7% of goal-extraction errors came from treating requirement-level contributions as outcomes.The remaining errors involved incorrect author attribution or implausible outcomes.
  • 61.5% of requirement-extraction errors came from treating artifact content as requirements.Other errors included response extraction, author attribution, minor details, and incorrect generation-time attribution.
  • 66.7% of influence-labeling errors involved incorrect author attribution, while 33.3% involved overly weak influence links.Weak links included entities sharing a broad topic without a strong or direct relationship.
  • Participants largely agreed with the tool’s analyses of goal hierarchy, requirements, and influence relations.The study used Likert ratings and participant explanations to assess alignment with their own conversations.
  • Participants identified interface confusion, unrepresented cognitive effort, and disagreement about whether indirect influence was meaningful.These concerns bound how users interpreted granular percentages and assistant contributions.

B.4 Sensitivity to LLM-as-a-Judge Errors

The sensitivity analysis tests whether controlled errors in judge outputs alter role-level contributions or preserve the paper’s key findings. Across simulated corruption rates up to 30%, all four findings remain intact.

  • 0%–30% controlled error rates were injected into judge outputs, with 300 trials run per error rate.Errors were independently sampled across action roles, requirement creators, goal flags, influence scores, and utterance attribution.
  • User values in the role-level contribution table are the negatives of the displayed values.This sign convention is needed when interpreting Table 5.
  • 100% of the four key findings were preserved across 300 trials at every tested error rate.Even at a 30% error rate, no finding fell below 50% preservation.

B.5 Using Multiple LLMs as Judges

The paper compares GPT-5.2 with alternative LLM judges using separate agreement measures for goal extraction and influence labeling. Agreement can diverge when judges segment requirements differently or assign origins differently.

  • Goal-extraction agreement is measured by mean Jaccard similarity over greedily matched action sets.The metric penalizes missed and over-claimed actions and is sensitive to segmentation granularity.
  • Llama-3.1-8B produces approximately 2.7 times as many requirements as GPT-5.2 during requirement extraction.This over-segmentation contributes to particularly low agreement between the judges.
  • Table 7 reports agreement with GPT-5.2, while Table 8 reports token usage and estimated cost by dialogue length and step.The tables separate judge agreement from computational accounting.
  • 77% versus 37% of requirements are attributed to assistant utterances by Llama-3.1-8B and GPT-5.2, respectively.The differing origin assignments further explain their low requirement-extraction agreement.
  • Influence-labeling comparisons hold GPT-5.2’s upstream outcomes, requirements, actions, and candidate pairs fixed.Alternative judges therefore perform only the downstream influence-labeling step.

D.1 Qualitative Examples

Qualitative analyses show that users and assistants shape goals through different action types and often contribute at different levels of requirement specificity. Assistants more often add implementation-oriented details, especially in technical tasks, while contributions overlap more in open-ended domains.

  • Actions users and assistants employ: Users primarily use Request, Constrain, and Instruct actions, whereas assistants more often use Suggest, Recommend, Provide, and Describe actions.The contrast separates explicit user goal shaping from advisory or execution-linked assistant contributions.
  • Task characteristics: Models generate more requirements in closed-ended tasks than in open-ended tasks.Figure 6 presents this comparison across task characteristics.
  • Requirement content: Assistant-generated requirements more often encode technical constraints, environmental assumptions, and correctness checks than user-generated requirements.The qualitative comparison uses PCA projections of requirement embeddings across four task types, without formal clustering.
  • Requirement content: In more open-ended domains, assistant- and user-generated requirements occupy largely overlapping semantic regions.The overlap suggests assistants often elaborate or concretize requirement types users could also introduce.
  • Manual categorization: The study samples 60 requirements—30 user-generated and 30 assistant-generated—for manual subtype categorization.The resulting proportions are reported in Tables 10–12, while Table 9 defines interaction subtypes and patterns.

E User Simulator Implementation

The simulations preserve free-form collaboration while varying interaction protocols and user guidance. They also evaluate COTRACE with paired human–LLM and human–human travel-planning sessions.

  • Interaction protocols: Agentic-CoGym lets the assistant message, act through structured tool calls, or wait, making communication optional before environment actions.Chat-CoGym instead requires a teammate-facing message before every tool call.
  • Interaction protocols: The interventions use soft behavioral guidance rather than rigid turn-by-turn rules, allowing interactions to proceed naturally.The steering and underspecified prompts vary collaborator behavior without dictating every response.
  • User guidance: The steering user varies autonomy delegation, open questions, continuation requests, added constraints, acknowledgments, and redirection across turns.The prompt also allows the user not to steer when the assistant is already progressing.
  • User guidance: The underspecified user expresses tentative preferences, hedges, reveals information gradually, and avoids volunteering detailed requirements unless asked.The prompt discourages fabricated details and encourages natural uncertainty.
  • Study implementation: The human study recruits 10 participants in five pairs for matched human–LLM and human–human travel-planning sessions, followed by tool inspection, surveys, and interviews.Session order is counterbalanced, and the two sessions use distinct destinations and contexts.

F.1.2 Responses

Participants described distinct collaboration experiences with chatbots and humans, while COTRACE helped reveal contribution patterns and hidden LLM work. Chatbots were valued for fast, convenient execution, whereas humans supported richer goal shaping and interpersonal validation.

  • Tool reflections: COTRACE helped participants notice previously unrecognized cognitive-reflection and hidden LLM work during collaboration.Participants reported that the tool made LLM constraints and guardrails more apparent.
  • Partner comparison: Chatbot collaboration was faster and stronger for execution, while human partners contributed more to collaborative goal shaping.Participants associated chatbots with logistical planning and humans with back-and-forth requirement development.
  • Partner comparison: Human partners could challenge, clarify, and validate proposals, while social constraints and partner preferences shaped human interaction.Participants described human dialogue as more conversational and attentive to others’ feelings.
  • Partner comparison: With chatbots, users were more direct, instrumental, and demanding; with humans, they were more considerate, open, and mentally engaged.Some users relied more on partners when lacking domain knowledge, while one participant reported similar behavior across settings.
  • Satisfaction: Participants were generally satisfied with chatbots because they met requirements, offered speed and convenience, and adapted to new constraints.Some participants nevertheless described responses as generic, questioned information reliability, or found chatbots weak for personal opinions.

F.2.2 Responses

COTRACE changed how evaluators interpreted students’ LLM reliance without changing grades. Goal-level summaries and visualizations made contribution and execution patterns faster and more detailed to assess than raw logs alone.

  • Assessment changes: Evaluators revised LLM-reliance ratings in three of four evaluator–student cases, but neither evaluator changed grading decisions.One evaluator identified additional code-explanation and writing-revision behaviors; the other revised both reliance ratings downward.
  • Assessment utility: Goal-level views made it easier and faster to determine whether students or the LLM shaped and executed each goal.Evaluators contrasted this with cumbersome raw-log review that could focus attention on only the interaction’s beginning.
  • Assessment utility: COTRACE exposed previously unnoticed details about code generation, execution, explanation requests, and text-revision assistance.It also showed evaluators that both students participated more actively in goal shaping than raw logs suggested.

G ShareChat Data Sampling

The ShareChat data were filtered to English conversations with at least four user–assistant turns, then labeled using chunk-level topic classification. Random or tangential chunks were excluded, while coherent chunks received taxonomy labels aggregated by majority vote.

  • Conversations were filtered to English samples containing at least 8 messages, equivalent to at least 4 user–assistant turns.
  • Single-topic chunks received exactly one label from the predefined taxonomy, and conversation-level topic labels were assigned by majority vote across chunks.
  • Long conversations were divided into non-overlapping chunks of N turns, such as N = 10 or 20, to improve labeling reliability.
  • Each chunk was classified as either single topic or random or tangential, with random or tangential samples discarded.
  • The labeling prompt specified the taxonomy, output schema, classification rules, and chunk text, while requiring JSON-only responses with no extra keys.
Loading 2605.21363v2…