PaperlayerView exact revision on arXiv

Source-linked AI summary

Alignment Games: A Framework for Conceptual Repair in Human-AI Collaboration

Hari Subramonyam, Maneesh Agrawala, Sean Follmer

arXiv:2609.35197v1cs.HCcs.AI

TL;DR

Users and AI systems can apply different task-relevant meanings to the same concept, while the system’s operative interpretation may remain implicit. Alignment Games combines a situated conceptualization model with a vocabulary of moves for surfacing and repairing those differences. The framework reframes collaboration around reaching a task-sufficient interpretation, but its proposed repair sequences have not been empirically established as effective.

  • Problem

    Generative AI systems’ implicit interpretations of concepts such as “clearer” or “more engaging” can leave users to infer and indirectly repair task-relevant mismatches.

    Source: §1 · p. 2

  • Method

    Alignment Games combines a task-relative situated-frame representation with structured moves that intervene on the situation, interpretation processes, or resulting frame.

    Source: §1 · p. 2, §7 · p. 16

  • Results

    The framework shifts the unit of analysis from output satisfaction to how collaborators reach a task-sufficient interpretation of what should be produced.

    Source: §6 · p. 13, §7 · p. 16

  • Takeaways & Limitations

    The framework offers HCI a vocabulary for making conceptual alignment more explicit and designable in generative interfaces.

    Source: §6 · p. 13

    The proposed move vocabulary is generative rather than exhaustive or empirically established, and the examples do not show that its repair sequences are optimal or improve diagnosis or repair effort.

    Source: §6.3 · p. 15

Abstract

from arXiv · show

The meaning of a concept in use is shaped by the situation, task, goals, and prior knowledge. For example, a request to make a poster "visually appealing for a five-year-old" might evoke bright colors and cartoon imagery for one collaborator, but less text, bold shapes, and visual simplicity for another. We call such task-relevant differences conceptual misalignment. We introduce Alignment Games, a framework for making these differences visible and repairable during human-AI interaction. Drawing on theories of situated conceptualization, we characterize task-specific conceptual frames in terms of relevant attributes, values, relations, constraints, and priorities. We then define alignment moves that intervene on the situation, the reasoning used to interpret it, or the resulting frame. Through examples from educational content generation, creative coding, and argumentative writing, we show how these moves can be composed into repair sequences and derive design principles for supporting task-sufficient conceptual alignment at runtime.

1 Introduction

Collaborators can use the same words yet interpret task concepts differently, leaving users to infer and repair an AI system’s implicit interpretation through output edits. Alignment Games makes those differences explicit and supports repair through structured moves and sequences.

1 Introduction

  • Task-relevant differences between collaborators’ situated conceptualizations can persist beneath shared language and concern which qualities matter, their values, relations, and priorities.The poster example shows that both bright colors and cartoon imagery, and reduced text and visual simplicity, can plausibly answer the same request.Source: §1 · pp. 1–2
  • When an AI’s operative interpretation remains implicit, users must infer the mismatch and repair it indirectly, making conceptual repair costly and uncertain.A prompt such as “make it less childish” may fix one dimension while leaving the underlying mismatch untouched or changing satisfactory aspects.Source: §1 · p. 2
  • Alignment Games makes conceptual repair explicit and designable by letting collaborators intervene on the interpretation that produced an artifact, not only on the artifact itself.A requester can clarify missing attributes, values, or priorities, or reframe the task.Source: §1 · p. 2
  • Structured sequences of alignment moves let collaborators diagnose, negotiate, and repair conceptual misalignment across situations, frame-construction processes, and resulting frames.The framework combines a process model of situated conceptualization with a grammar of moves for those intervention loci.Source: §1 · p. 2

2 A Note on Framework Development

The authors developed Alignment Games through iterative theory-building that combined literature synthesis, conceptual modeling, and analysis of existing human–AI interfaces.

2 A Note on Framework Development

  • More than 200 hours of collaborative analysis and discussion, alongside weekly all-author meetings, compared alternative frame representations, misalignment sources, and candidate repair moves.The authors tested representations against breakdowns and mechanisms reported in prior HCI systems, refining constructs when they collapsed distinct phenomena or failed to characterize plausible repairs.Source: §2 · p. 2

3 Situated Conceptualization and Conceptual Misalignment

Situated conceptualizations are constructed from external situations, internal task context, and prior knowledge through iterative processes that yield task-relative frames. The framework treats consequential differences between frames—not identical representations—as the alignment concern.

3.1 What Shapes a Situated Conceptualization?

  • A situated conceptualization draws on the external situation, internal task context, and prior knowledge, while the section’s process model focuses on inputs, processes, and representations relevant to repairing conceptual differences.Goals, roles, values, and resource constraints can affect interpretation, as can collaborators’ different experiences and expertise.Source: §3 · p. 2, §3.1 · pp. 3–4

3.2 How Is a Situated Conceptualization Constructed?

  • Perception and attention, retrieval, reasoning, and simulation iteratively organize situation features and prior knowledge according to the task context.The same visual feature can be judged differently depending on the task, and retrieval selectively supplies candidate attributes, values, and prior frames.Source: §3.2 · p. 4, §3.2.1 · p. 4, §3.2.2 · p. 4

3.2.2 Retrieval.

  • Reasoning organizes relevant attributes, their values and relations, and the constraints or priorities that govern a coherent interpretation.It can also reorganize conceptual structure through analogy, decomposition, abstraction, or blending.Source: §3.2.3 · p. 4

3.2.3 Reasoning.

  • Simulation lets people evaluate interpretations in imagined situations, while acting on an emerging interpretation changes the artifact and can prompt further revision.Imagining a child viewing a poster across a hallway may foreground legibility; new perceptual evidence can lead to reconsidered constraints or possibilities.Source: §3.2.4 · p. 4

3.3 What Does the Resulting Conceptual Frame Represent?

  • A situated conceptual frame organizes task-relevant attributes and values, their relations and constraints, preferred directions, relative importance, and broader conceptual framings.These elements form a coherent configuration rather than a set of independent parameters.Source: §3.3 · pp. 4–5
  • The frame is a task-relative abstraction for collaborative action and repair, not a complete account of cognition or a claim to expose an AI’s latent state.For AI systems, it is an operational representation of the current task interpretation, without assuming human-like conceptual representations.Source: §3.3 · p. 5

3.4 When Are Collaborators Conceptually Misaligned?

  • Conceptual misalignment is divergence between collaborators’ task-relevant frames, including differences that remain latent beneath shared words and those that propagate across related frame elements.Divergence can concern attributes, values, relations, constraints, priorities, or broader framing, and frames can change as collaborators encounter new information.Source: §3.4 · pp. 5–6

3.5 When Is Conceptual Alignment Sufficient?

  • Alignment is task-sufficient when remaining frame differences do not affect compatible judgments or coordination, even if collaborators’ representations are not identical.A small disagreement about a critical safety requirement or priority can matter more than larger differences in peripheral associations.Source: §3.5 · p. 6

4 Alignment Games: How Can Collaborators Repair Conceptual Misalignment?

Alignment Games shift coordination from understanding words or sharing a referent to inspecting and repairing task-specific conceptual frames. They organize repair as iterative interactions and distinguish moves by where they intervene and what the task requires.

4 Alignment Games: How Can Collaborators Repair Conceptual Misalignment?

  • Alignment Games shift coordination from whether an utterance is understood to the task-specific conceptualization guiding action.Collaborators can share a referent and understand the same words while differing over relevant attributes, values, relations, or priorities.Source: §4 · p. 6

4.1 What Is an Alignment Game?

  • An Alignment Game is a structured, multi-turn interaction for diagnosing, negotiating, repairing, and validating task-relevant differences between situated frames.Evidence may be direct or indirect, and moves can be linguistic or enacted through examples, comparisons, or artifact changes.Source: §4.1 · p. 6
  • Uptake—including rejection, counterproposal, or partial enactment—provides new evidence for frame updates, and play continues until alignment is task-sufficient or repair is abandoned.Convergence may involve one or both collaborators changing; peripheral differences can remain.Source: §4.1 · pp. 6–7, Figure 4 · p. 8

4.2 What Is an Alignment Move?

  • An alignment move is defined by its intended effect on a situated frame and can be characterized by function, locus, target, scope, and expected effect.Its success also depends on whether the partner can recognize and take it up, and some moves have preconditions.Source: §4.2 · pp. 7–8

4.2.1 Situation-Level Moves: Changing the Basis for Conceptualiza-

  • Situation-level moves change the evidence or task context used to construct a frame, for example by clarifying a goal, constraint, or relevant part of an artifact.They alter the basis for interpretation rather than directly specifying how the partner’s frame should change.Source: §4.2.1 · p. 9

4.2.2 Process-Level Moves: Changing How the Frame Is Constructed.

  • Process-level moves redirect attention, retrieval, decomposition, analogy, or simulation to change how available information is construed.A simulation can make attributes or constraints salient without prescribing a direct frame edit.Source: §4.2.2 · p. 9

4.2.3 Frame-Level Moves: Directly Revising the Conceptual Frame.

  • Frame-level moves directly revise attributes, values, dependencies, constraints, priorities, or broader framing, sometimes requiring coordinated downstream changes.Move families are complementary, and classification follows the intervention locus rather than all its consequences.Source: §4.2.3 · p. 9

4.3 What Factors Inform Move Selection and Repair Effort?

  • Move selection depends on diagnostic specificity, task significance, and repair effort; effective repair reaches a task-sufficient state with minimal unnecessary change.A small disagreement about a critical constraint may matter more than a large peripheral difference, while broad reframing can alter many frame elements.Source: §4.3 · p. 9, §4.3.1 · p. 9, §4.3.2 · p. 9

5 Alignment Games in Existing Human–AI Interfaces

The paper applies Alignment Games to reported breakdowns in educational content generation, creative coding, and argumentative writing. These are analytic reinterpretations that sketch possible repair sequences, not evaluations of the original systems or claims about their actual internal representations.

5 Alignment Games in Existing Human–AI Interfaces

  • The examples reinterpret reported breakdowns in three HCI systems and sketch possible alignment moves rather than evaluating the systems or asserting their actual internal representations.They ask where a consequential conceptual difference might lie and which interface affordances could support repair.Source: §5 · p. 11

5.1 Salience Alignment Game

  • In quiz generation, a possible mismatch concerns which features make selected text important; presenting, pruning, simulating, and weighting attributes could negotiate instructional salience.The proposed sequence shifts from repeatedly regenerating questions to directly negotiating the operative interpretation of importance.Source: §5.1 · p. 11

5.2 Conceptual Blend Alignment Game

  • In sketch blending, surprising or incoherent merges may reflect differing views of which source properties should carry over and how they should relate.Presenting frames, mapping an intended analogy, proposing correspondences, and evaluating previews makes alternative mappings inspectable before commitment.Source: §5.2 · pp. 12–13

5.3 Epistemic Grounding Game

  • In argumentative writing, inadequate evidence suggestions may reflect different constraints, evidence values, or priorities; teaching a reasoning structure and decomposing evidence can revise that frame.The proposed repair changes the conceptual structure and constraints used to interpret evidence, rather than merely selecting a different value.Source: §5.3 · p. 13

6 Discussion

Alignment Games shifts attention from output success to how collaborators form a task-sufficient interpretation of what should be produced. It proposes design principles for making conceptual differences visible, repairing them, and negotiating task-specific meaning at runtime.

6.1 Design Principles for Alignment Games

  • Interfaces should make operative interpretations inspectable and revisable, exposing task-relevant attributes, values, relations, constraints, priorities, and broader framings.These frames are task-relative interaction objects, not faithful readouts of model cognition; existing systems expose different parts of them.Source: §6.1 · p. 13
  • Repair should proceed through composable trajectories: explore uncertain mismatches, make targeted changes, validate uptake, and preserve reversible or branchable paths.Previews, reversible edits, branching alternatives, and indications of affected frame elements can reduce the risk of consequential changes.Source: §6.1 · pp. 13–14
  • Systems can redistribute alignment work by making interpretations observable, easing likely interventions, and potentially initiating repair when uncertainty or conflicting evidence arises.This can shift diagnostic work away from users; the framework does not require human-AI collaboration to be symmetric.Source: §6.1 · pp. 14–15
  • Interfaces should support exploring alternatives and preserving useful differences, aligning only on aspects that matter for the current joint activity.Conceptual differences may reveal possibilities neither collaborator would have produced independently.Source: §6.1 · p. 15

6.2 Conceptual Alignment as Runtime Alignment

  • Conceptual alignment is runtime negotiation of situated meaning, since locally important concepts depend on the situation, goals, prior decisions, and collaborators.Alignment is an ongoing accomplishment as tasks change and new evidence becomes available, complementing design- and training-time approaches.Source: §6.2 · p. 15

6.3 Limitations

  • Operational frames are partial constructions: people’s conceptualizations are only partly accessible, and AI explanations need not faithfully reveal internal processing.Future work should investigate elicitation, inference, revision, and evaluation without requiring users to specify every aspect of their understanding.Source: §6.3 · p. 15
  • The proposed move vocabulary is generative rather than exhaustive or empirically established, and examples do not establish that inferred misalignments caused reported breakdowns.Empirical work is needed to assess move selection, effectiveness under different uncertainties, and whether interfaces reduce repair effort.Source: §6.3 · p. 15
  • Making frames observable risks enabling detailed user models and unwarranted attributions of AI agency; systems should clarify representation status and give users control over stored, reused, or shared information.The framework addresses task-situated meaning, not broader safety, fairness, cultural values, or societal alignment.Source: §6.3 · pp. 15–16

7 Conclusion

Alignment Games combines a situated-conceptualization model, task-relative frames, and composable alignment moves to make task-relevant differences visible and actionable during interaction.

7 Conclusion

  • Alignment Games combines a situated-conceptualization process model, a task-relative frame representation, and alignment moves spanning the situation, conceptualization processes, and resulting frame.The framework positions alignment as a runtime collaborative process for reaching task-sufficient mutual understanding.Source: §7 · p. 16
  • Conceptual misalignment arises when collaborators construct different task-relevant interpretations of concepts guiding their joint activity.Differences may remain hidden beneath shared language or reflect different concepts, attributes, relations, or priorities.Source: §7 · p. 16
  • The framework provides a foundation for designing AI systems that help people shape the interpretations guiding generation.Source: §7 · p. 16

A Determinants of Conceptual Misalignments

Conceptual misalignment can arise from differences in situational cues, task context, prior knowledge, expertise, acquisition, culture, and concept properties.

A Determinants of Conceptual Misalignments

A.1 Variability of Conceptual Inputs

The grain size and tangibility of a situation influence which elements collaborators consider relevant and how much shared grounding they have.

A.1 Variability of Conceptual Inputs

  • Broader situational grain sizes increase uncertainty about which elements or concepts should be activated.Grain size refers to a situation’s spatial and temporal extent.Source: §A.1.1 · p. 19
  • Real-life situations provide perceptual grounding and shared reference points, whereas hypothetical or counterfactual scenarios lack those anchors.Source: §A.1.1 · p. 19
  • Without perceptual anchors, hypothetical or counterfactual scenarios require more inference and increase variability in collaborators’ conceptual frameworks.Source: §A.1.1 · p. 19

A.1.1 Situational Variance.

Familiarity, psychological distance, goals, affect, and prior experience shape the features, abstractions, and values people bring to a task.

A.1.1 Situational Variance.

  • Familiar situations activate established knowledge and scripts, reducing cognitive effort but risking rigidity through retrieval-based processing and functional fixedness.Source: §A.1.1 · p. 19
  • Greater psychological distance encourages abstract construal, while proximity encourages concrete reasoning; collaborators differing in distance or familiarity may therefore diverge in salient features or goals.Source: §A.1.1 · p. 19
  • Goals regulate perception and conceptual activation, so differing or underspecified goals can yield misalignment; long-term goals favor abstract representations, while immediate goals favor concrete ones.Source: §A.1.2 · p. 19
  • Positive moods broaden category inclusion and global processing, whereas negative moods heighten focus on local, detailed features.Source: §A.1.2 · p. 19
  • Prior experience shapes encoded features, their relations, and typical values, depending on whether concepts are learned through prototype similarity, exemplar similarity, or causal theory-building.Source: §A.1.3 · p. 19

A.1.3 Prior Knowledge.

Differences in experience, learning, culture, and expertise shape how collaborators structure and retrieve concepts. These differences can persist even when they use the same words.

A.1.3 Prior Knowledge.

  • Different experienced examples, instruction, ideals, and expertise shape which attributes and values people associate with concepts.Expertise can refine conceptual structure, but deep expertise may also reduce flexibility in reappraisal.Source: §A.1.3 · p. 19
  • Perceptual learning supports richer action-oriented simulations, while age and social or cultural learning affect concepts’ stability, flexibility, and alignment within communities.Early-learned concepts tend to be retrieved more quickly and remain more stable; later-acquired concepts remain more flexible.Source: §A.1.3 · p. 19
  • People differ in the episodic and sensory experiences that activate concepts, as well as in the organization of their semantic networks.Flexible, associative networks can support divergent thought and reinterpretation.Source: §A.1.3 · pp. 19–20

A.2 Conceptual Sources of Variance

Concepts vary in structure and use, shaping how collaborators interpret the same terms. Concrete concepts tend to be more perceptually grounded, while abstract concepts allow greater variation.

A.2 Conceptual Sources of Variance

  • Conceptual variability shapes how collaborators interpret and apply the same terms.The section contrasts concrete concepts such as “door” and “chart” with abstract concepts such as “love” and “beauty.”Source: §A.2.1 · p. 20
  • Concrete concepts tend to elicit stronger perceptual grounding and less variation in associated attributes than abstract concepts.Abstract concepts rely more on social and introspective contexts and exhibit broader variability.Source: §A.2.1 · p. 20
  • Concrete concepts are more imageable and tied to sensory experience, supporting richer perceptual simulation.Source: §A.2.1 · p. 20

A.2.1 Frame/Concept Variance.

Conceptual misalignment can arise from differences in concepts, attributes, values, perception, retrieval, reasoning, and communication. These differences shape how collaborators frame situations and interpret shared language.

A.2.1 Frame/Concept Variance.

  • Concepts vary in abstraction, contextual embedding, associative structure, and semantic richness, creating different interpretive latitude.Concrete concepts tend to be perceptually grounded; abstract concepts and concepts with low situational systematicity can evoke more variable contexts.Source: §A.2.1 · p. 20

A.2.2 Attribute Variance.

  • People select attributes through prototypes, ideals, and causal or normative relations, so collaborators can foreground different features of the same concept.Source: §A.2.2 · p. 20

A.2.3 Value Variance.

  • Collaborators may assign different defaults and acceptable ranges to shared attributes, or prioritize competing goals and constraints differently.Broader acceptable ranges can support flexibility but risk misalignment when precision is required.Source: §A.2.3 · pp. 20–21

A.3 Process Variance

  • Differences in perceptual sensitivity, expertise, and goals affect which features collaborators notice and make salient.Joint attention can be disrupted under high attentional load; visual cues such as gaze can help signal focus and relevance.Source: §A.3.1 · p. 21

A.3.2 Retrieval variance.

  • Concept retrieval varies with use frequency and recency, situational relevance, encoding strength, and interference from competing information.Source: §A.3.2 · p. 21

A.3.3 Reasoning Variance.

  • Different reasoning strategies and conceptual structures can produce distinct interpretations, including internally consistent but different mappings for the same analogy.The order of information processing can affect analogy mappings, and causal relations can shape conceptual combination.Source: §A.3.3 · p. 21

A.4 Asymmetry of thinking between agents

  • Collaborators may differ in how deeply they reason, so one may use deliberate analysis while another relies on faster, heuristic judgments.Under time pressure or limited expertise, people may use cognitively underspecified models rather than full mental models.Source: §A.4 · p. 21

A.5 Communication: Compression, Specificity, and Noise

  • Language compresses rich conceptual frames into limited, sometimes ambiguous or vague expressions, leaving listeners to infer omitted meaning.When common ground is weak, those inferences can diverge; polysemy can also activate the wrong sense of a word.Source: §A.5 · pp. 21–22
Loading 2609.35197v1…