Who Takes the Lead? Model Ability and Task-Conditioned Collaboration with Humans

Motivation

• Falling human involvement may reflect a correct division of labor rather than a collaboration failure.The paper frames collaboration as allocating control in response to the partners’ relative task advantages.Source: §1 · p. 1

Research question

• The study asks how an agent’s standalone task capability relates to when and how it involves a human collaborator..Source: §1 · p. 1

Motivation

• Human-involvement rates alone cannot distinguish selective use of an advantaged partner from an agent’s limited independent capability.The paper therefore pairs a solo baseline with a task where the human’s marginal value is varied by construction.Source: §1 · p. 2

Approach

• The shared-control maze holds the team objective and interaction protocol fixed while initial avatar positions vary relative shortest-path advantage.The model can act itself, recommend a human move, or hand over without a recommendation.Source: §1 · p. 2

Results

• Near-ceiling solo models acted for themselves when routes were tied or favored the agent, but involved humans on 77.5% to 82.5% of initial decisions when the human route was shorter.Command R showed no comparable selectivity and maintained similar whole-game action composition across relative-advantage states.Source: §1 · pp. 2–3

Human–AI complementarity and deferral

• In sequential collaboration, control allocation can recur throughout an episode, and the value of involving a partner can change with the task state.Prior work often studies prediction-time routing under a learned deferral policy.Source: §2 · p. 3

Appropriate reliance

• Aggregate reliance measures can obscure whether participation reflects appropriate use of a teammate or limited independent capability.Prior work finds reliance frequency insufficient to characterize collaboration quality and acceptance of AI advice does not necessarily improve team outcomes.Source: §2 · p. 3

Agent-side selective involvement

• This evaluation combines a solo capability baseline with controlled variation in a human teammate’s positional advantage to assess task-conditioned involvement.Related work studies intervention, help-seeking, and coordination through other mechanisms.Source: §2 · p. 3

3 Human–Agent Shared Maze

• Either avatar reaching the common target ends the episode, and the team seeks to reach it in as few turns as possible.Only one avatar moves per team turn; initial positions change the human’s task-specific utility while the maze and objective remain fixed.Source: §3 · pp. 3–4

3.1 Maze and State Space

• Mazes are connected spanning trees with a unique simple path between every pair of cells; the target is placed at a diameter endpoint to widen feasible route lengths.Start cells are sampled to realize requested distances exactly; infeasible distance pairs trigger up to 200 deterministic layout attempts.Source: §3.1 · p. 4

3.2 Observation Space and Interaction History

• The agent receives the full maze topology, target, both avatar positions, and a text history of resolved interactions; the human acts from a local view.The history records actor, action type, recommendation, executed displacement, and whether a human followed or overrode a recommendation.Source: §3.2 · p. 4

3.3 Action Space

• The model chooses self-action, guided human action with a directional recommendation, or human deference without a recommendation.Guidance lets the human follow or override the recommended direction; deference leaves direction entirely to the human.Source: §3.3 · p. 5

3.4 Task-Conditioned Comparative Advantage

• The human’s positional comparative advantage is the agent’s shortest-path distance minus the human’s, with positive, negative, and zero values marking human advantage, agent advantage, and equal routes.These oracle distances vary task geometry while remaining unstated in the model prompt.Source: §3.4 · p. 5

3.5 Transition Dynamics and Termination

• Each turn executes one avatar movement, records the resolved state in the history, and ends when either avatar reaches the target or the maximum horizon is reached.Source: §3.5 · p. 5

4.1 Model Capability Baseline

• GPT 5.6 Sol finished every solo maze optimally, Command A+ completed every episode with occasional detours, and Qwen timed out once; Command R completed 22% with MDO 0.53.With four models, the capability-group comparison is descriptive, not a continuous scaling relationship.Source: §4.1 · pp. 5–6, Table 1 · p. 6

4.2 Co-play Design and Measures

• Four authors completed 480 co-play episodes across four models and three initial route conditions, with 40 episodes per model–condition cell and a ten-turn horizon.Initial route pairs were (5, 7), (5, 5), and (7, 5) for agent-favored, tied, and human-favored conditions.Source: §4.2 · p. 6
• The primary outcome was the first allocation decision; secondary measures covered actor-aligned allocation, directional optimality, and team success and efficiency.The study reports 95% percentile intervals from 1,000 two-way cluster-bootstrap samples, while comparisons remain descriptive given four participants.Source: §4.2 · p. 6

4.3 Results

• The three near-ceiling models shifted from near-universal self-action on tied or agent-favored routes to 77.5%–82.5% human involvement when the human route was shorter; Command R showed no positive selectivity.Table 2 reports initial control allocation; its caption defines positive ∆0 as a shorter human route.Source: §4.3 · p. 6, Table 2 · p. 7
• Command A+ completed 97.5%, 100%, and 87.5% of episodes in agent-favored, tied, and human-favored conditions; Command R completed 17.5%, 22.5%, and 45.0%, respectively.Qwen 3.8 27B and GPT 5.6 Sol completed every episode by turn ten across all three conditions; assigned route advantage does not identify a causal effect of human involvement on completion.Source: §4.3 · pp. 6–8
• Human involvement occurred overwhelmingly through guidance rather than deference, and no model chose deference on an initial decision in any condition.Whole-game deference shares were zero or near zero across models and conditions.Source: §4.3 · p. 8
• Across observed turns, the three near-ceiling models shifted toward guidance when the human was closer, while Command R stayed near 35–40% self-action and 60–65% guidance across advantage states.Figure 3 groups turns by current route advantage; because prior actions may affect that advantage, these comparisons are descriptive rather than causal.Source: Figure 3 · p. 8, §4.3 · p. 8

5 Conclusion and Discussion

• GPT 5.6 Sol, Command A+, and Qwen 3.8 27B achieved near-ceiling solo performance, while Command R completed substantially fewer solo episodes.This baseline helps distinguish selective involvement from involvement associated with limited independent task capability.Source: §5 · p. 9
• Near-ceiling-capability models mostly chose self-action when the AI route was shorter or tied, but increased human involvement when the human route was shorter.Command R, with weaker solo performance, did not show the same positive allocation pattern; human involvement was primarily guidance rather than full deference.Source: §5 · p. 9
• Aggregate involvement rates alone can obscure whether an agent responds to a human’s comparative advantage or involves the human despite limited independent capability.Capability baselines and condition-specific allocation measures provide complementary information about how control is distributed.Source: §5 · p. 9

6 Limitations and Future Work

• The study evaluates only four models, few human participants, and one maze environment and interaction protocol.Its comparisons are descriptive of the evaluated models and setting, and establish an association rather than the direction of a relationship.Source: §6 · p. 9
• Establishing the generality and direction of the capability–selectivity relationship requires broader model samples, participant populations, task domains, and collaboration protocols.Future work proposes extending the analysis beyond navigation to cooperative games, everyday work, and collaborative coding.Source: §6 · p. 9
• The measures are described as transferable to shared tasks where partner advantage can be manipulated by construction.The proposed extensions include settings where human comparative advantage comes from sources other than navigation.Source: §6 · p. 9

7 Ethical Considerations

• The maze is a simplified controlled environment, so the findings should not justify reducing human oversight or decision authority in consequential real-world settings.Source: §7 · p. 9

A Prompts and History Extraction

• The prompts use a teammate framing and interaction history, while withholding oracle distances, relative advantage, and the remaining turn budget.The prompt families differ for solo and co-play, and both use adjacency-list state rendering.Source: A Prompts and History Extraction · p. 14

A.1 Observation Extraction

• The observation includes the wall grid, target, avatar positions, and legal moves, but excludes oracle distances and who is closer.The state is rendered as tagged adjacency-list text with coordinates, origin, target, teammate position, and legal-move sets.Source: §A.1 · p. 14

A.2 History Extraction

• The next prompt’s history records model actions and human moves, including whether a human followed or overrode guidance.Unparseable replies or illegal moves trigger a clarification turn with the output contract and relevant legal moves restated.Source: §A.2 · pp. 14–15

A.3 Solo Prompts

• Solo prompts ask the model to navigate to the exit using the shortest route and output one legal move per turn.The model sees the full maze layout; the legal-move list is supplied for the current avatar.Source: §A.3 · p. 15

A.4 Co-play Prompts

• In co-play, either avatar reaching the exit ends the shared task, and each model turn selects self-movement, teammate guidance, or deference.Guidance is a recommendation the teammate may follow or override; deference hands over the turn without advice.Source: §A.4 · p. 15

B Model Access, Licensing, and Compute

• The evaluation includes GPT 5.6 Sol, Command A+, Qwen 3.8 27B, and Command R 08-2024.Source: §B.1 · p. 16
• GPT 5.6 Sol was accessed through the OpenAI API, while Command A+ and Qwen 3.8 27B open-weight releases use Apache 2.0.Source: §B.1 · p. 16
• Command R 08-2024 is released under CC-BY-NC 4.0 with the Cohere Labs Acceptable Use Addendum.All models were used subject to applicable licenses and provider terms.Source: §B.1 · p. 16

• For hosted APIs, provider-side hardware specifications were unavailable, and reported costs reflect inference or compute-rental costs.Source: §B.2 · p. 16
• Table 3 covers inference settings and approximate compute expenditure across all reported experiments.Source: Table 3 · p. 16
• The Cohere cost combines Command R and Command A+ expenditures.Source: Table 3 · p. 16

C Metric Reference and Detailed Results

• Table 4 expands evaluation abbreviations; initial rates use first decisions across episodes, while whole-game shares average within-episode shares across episodes.In the solo evaluation, S@k, RCT10, E, and MDO concern the model's single avatar; E uses its oracle route length.Source: C Metric Reference and Detailed Results · p. 17, Table 4 · p. 17

C.1 Solo Capability

• Solo capability intervals resample maze seeds across 1,000 replicates, preserve pairing across 50 shared seeds, and treat horizon timeouts as valid failures.Table 6 reports invalid-move, unparseable, and retry rates over 1,230 recorded LLM steps; unparseable outputs were not observed, and Command R produced the sole invalid move.Source: §C.1 · p. 17, Table 5 · p. 18, Table 6 · p. 18

C.2 Initial Control Allocation

• Tables 7 and 8 report two-way cluster-bootstrap 95% confidence intervals for first-action estimates and claim-focused estimates, respectively.Each condition contains 40 episodes for the first-action estimates; Table 8 supplies definitions by reference to Table 4.Source: Table 7 · p. 18, Table 8 · p. 18

C.3 Whole-Game Collaboration and Performance

• Table 10 reports team outcome estimates with 95% confidence intervals by model and initial route advantage; lower RCT10 and higher E are better.Each condition contains 40 episodes.Source: Table 10 · p. 19
• Figure 4 plots cumulative episode completion by team turn for AI-shorter, equal-route, and human-shorter starting conditions, with 40 episodes per model-condition cell.Source: Figure 4 · p. 20
• Figure 4 marks the turn-7 checkpoint; episodes unfinished at turn 10 remain incomplete.Source: §C.3 · p. 19, Figure 4 · p. 20