Position: Towards Bidirectional Human-AI Alignment

• Specification gaming arises when incomplete objectives or feedback lead designers to rely on proxies such as human approval, allowing apparently correct decisions for opaque or wrong reasons [8] [11].Source: §1 · pp. 1–2
• Scalable oversight is difficult because evaluating complex AI behavior can be slow or infeasible, increasing the challenge of aligning systems through human feedback [5] [13].Source: §1 · p. 2
• As AI advances, alignment must adapt to changing human values and account for long-term cognitive and social impacts through ongoing, cross-disciplinary work.Source: §1 · p. 2
• Traditional alignment treats shaping AI toward desired outcomes and avoiding harm as a static, one-way process, while human-AI interaction creates evolving feedback loops.Source: §1 · p. 2
• The authors argue that researchers should explicitly reflect on alignment and account for bidirectional, dynamic human-AI interactions to support responsible and safe systems.Source: §1 · p. 2
• A systematic review of over 400 papers across HCI, NLP, ML, and related fields informs the proposed Bidirectional Human-AI Alignment framework.Source: §1 · p. 2
• The framework adds “Aligning Humans to AI,” encompassing cognitive, behavioral, and societal adaptation to rapid AI advancement.Source: §1 · p. 2
• Existing research has gaps in human value modeling, oversight of model inference, critical evaluation of embedded AI values, and broader societal impact.Source: §1 · p. 2

• The authors formally propose Bidirectional Human-AI Alignment based on their systematic review analysis.Source: §2 · p. 2
• The paper adopts human values—what individuals or society morally expect AI to do—as its alignment objective, while noting that goals such as intentions and preferences are often used interchangeably.Source: §2 · p. 2
• Stakeholders include end users, practitioners, and organizations, whose values may conflict; the paper adopts pluralistic value alignment rather than a universal moral theory [28].Source: §2 · pp. 2–3
• The paper selects Schwartz’s Theory of Basic Values for its cross-cultural applicability, contextual relevance, and frequent use in NLP research [38, 39, 40, 41].Source: §2 · p. 3
• Bidirectional Human-AI Alignment comprises two connected processes: aligning AI with humans and supporting humans’ agency, critical thinking, collaboration, and societal adaptation around AI.Source: §2 · p. 3

3 Bidirectional Human-AI Alignment Framework

• The framework links aligning AI to human values with humans’ cognitive and behavioral adaptation to AI in a feedback loop.It frames the two directions as interconnected rather than independent.Source: §3 · p. 3

3.1 Align AI to Humans

• Human values are categorized by source and type, while specification approaches include explicit, implicit, and simulated feedback.The sources are individual, social, and interactive; the four value types are Self-Enhancement, Self-Transcendence, Conservation, and Openness to Change.Source: §3.1 · p. 4
• Values can enter AI development through instruction data, model learning, and inference-stage techniques, or through customization for users and communities.Evaluation includes human judgment and automated methods, but the review finds limited customization and a lack of standardized criteria for human-in-the-loop methods and platforms.Source: §3.1 · pp. 4–5
• The review highlights a strong focus on explicit value annotations, with implicit expressions, interactive value specification, and human-in-the-loop evaluation less studied.It also identifies underexploration of methods for customizing AI to individual or group values.Source: §3.1 · pp. 4–5

3.2 Align Humans to AI

• The review organizes human cognitive adjustment around learning to understand AI and critically assessing its outputs and behavior.It covers AI literacy and explanations, as well as trust and reliance, selective adoption, ethical auditing, and recalibrating expectations; the review finds gaps in training and critical evaluation across human values.Source: §3.2 · pp. 5–6
• Human adaptation research covers collaboration, AI’s effects on people and society, and evaluation of those effects at individual and societal scales.The review finds limited study of collaboration with AI that surpasses human capabilities and of AI’s evolving impacts over time.Source: §3.2 · p. 6

4 Underexplored Research Gaps and Challenges

• Figure 2 counts papers relevant to each dimension within the papers relevant to each research question.Gray bars show the research-question group; color bars show counts for its dimensions.Source: §4 · p. 6, Figure 2 · p. 7

Underexplored Dimensions in Aligning AI with Humans

• Implicit and simulated value feedback, inference-time or interactive alignment, and human-in-the-loop evaluation receive limited attention.The review notes that richer signals may provide context-sensitive information about values, while real-time adaptation could support dynamic or personalized contexts.Source: §4 · pp. 6–7

Underexplored Dimensions in Aligning Humans with AI

• Human-centered research emphasizes explanations but underexplores AI literacy, collaboration with highly capable AI, ethical auditing, and societal impacts.The review also identifies limited research on effects such as changes in human behavior, relationships, and public responses.Source: §4 · p. 7

5 Near to Long-term Risks and Opportunities

• The paper identifies specification gaming, dynamic co-evolution, and safeguarding co-adaptation as three alignment challenges.It presents them as near- to long-term research objectives.Source: §5 · p. 7

5.1 Specification Game

• Specification work should aggregate diverse, changing human values and develop ways to represent them for AI, including unstructured descriptions and behavioral data.The paper also recommends interactive interfaces that elicit nuanced, contextual values and use implicit signals.Source: §5.1 · p. 8

5.2 Dynamic Co-evolution of Alignment

• Alignment should continuously update AI as human values and society evolve, potentially forecasting changes and preparing flexible prompting or interventions.The paper frames this as co-evolution among AI, humans, and society.Source: §5.2 · p. 8
• People need ways to interpret and validate AI outputs, including when AI exceeds their abilities, and research should examine how people learn from AI and how society changes.Suggested directions include collaborative validation and scalable tools, alongside sustained study of effects on behavior, relationships, and governance.Source: §5.2 · pp. 8–9

5.3 Safeguarding Co-adaptation

• Safeguarding co-adaptation calls for interpretable and controllable AI actions, intervention mechanisms, and reliable ways to assess AI honesty.Recommendations include corrigible mechanisms, override protocols, training simulations, real-time dashboards, anomaly detection, and scalable monitoring tools.Source: §5.3 · p. 9

6 Limitations

• The sampled and filtered papers do not constitute an exhaustive collection of human-AI alignment research.The literature spans diverse venues and domains and is rapidly growing.Source: §6 · p. 9
• The surveyed papers and team members primarily focus on computing fields such as ML, NLP, and HCI.The paper notes that alignment research also involves cognitive science, psychology, and science and technology studies.Source: §6 · p. 9
• The authors describe the framework as extendable to disciplines beyond the fields primarily represented in their review.They present it as a foundational reference for future researchers despite the scope limitation.Source: §6 · p. 9

7 Conclusion

• The Bidirectional Human-AI Alignment framework treats alignment as two interdependent directions: aligning AI with human values and aligning humans with AI.The latter direction concerns helping people understand, evaluate, and adapt to AI systems.Source: §7 · p. 9
• The analysis of over 400 papers identifies limited support for long-term interaction, underdeveloped models of human values, and challenges in mutual intelligibility.Source: §7 · p. 9
• The paper concludes with three central challenges and actionable recommendations for reciprocal, robust, and context-aware human-AI alignment.Source: §7 · p. 9

8.1 Systematic Literature Review Process

• The review followed PRISMA-guided stages to identify, screen, and assess papers before coding the final corpus.Figure 3 presents the selection and refinement workflow and counts excluded records at identification, screening, and eligibility.Source: Figure 3 · p. 10, §8.1 · p. 10

8.1.1 Identification and Screening with Keywords.

• From AI-related domain venues, researchers retrieved 34,213 papers in the initial Identification stage, then screened for papers whose title or abstract contained at least one listed keyword, including 2,136 papers in the Screening stage.The search covered AI-related venues, primarily from January 2019 through January 2024.Source: §8.1.1 · p. 10

8.1.2 Assessing Eligibility with Criteria

• Eligibility review applied six research questions and produced a final corpus of 411 papers for detailed qualitative coding.Source: §8.1.2 · p. 10

8.1.3 Qualitative Code Development.

• Two authors coded each paper using an iteratively revised codebook, combining inductive and deductive coding across HCI and NLP/ML teams.The reported joint probability of agreement for annotations was 0.78.Source: §8.1.3 · pp. 10–11, §8.3 · p. 11

8.1.4 Framework Development and Rigorous Coding.

• The team developed the framework by integrating annotations, iterating through discussion and advisor feedback, and independently coding papers against the finalized framework.Additional AI ethics and related-domain papers were reviewed to strengthen coverage.Source: §8.1.4 · p. 11

8.2 Venues

• The review primarily covered 2019–January 2024 papers from HCI, NLP, and ML venues, then supplemented its collection with AI ethics and related work.The team also added missing codes and papers identified through FAccT, AIES, and related work.Source: §8.2 · p. 11

8.3 Keywords

• Keyword screening covered terms for humans, AI, LLMs, alignment, values, trust, interaction, visualization, explanation, evaluation, feedback, and ethics.Source: §8.3 · pp. 11–12

8.4 Inclusion and Exclusion Criteria

• The review selected papers against six research questions and specified inclusion and exclusion criteria, while acknowledging that its human-centered review of key studies is not exhaustive.Its scope excluded, among other areas, non-text modalities and studies without human-AI interaction.Source: §8.4 · pp. 12–13

10.1 A Comprehensive Taxonomy of Human Values

• The taxonomy maps values from alignment literature onto Schwartz’s theory, adding “Desired Values for AI Tools,” which includes usability and human-likeness.It organizes five high-order and 12 motivational value types across individual, societal, and interactional sources.Source: §10.1 · p. 14, Table 3 · p. 14

10.2 Insights into Human Values for Alignment

• Preference-based alignment may prioritize majority values and neglect marginalized groups, so research should address competing value priorities.Source: §10.2 · p. 15
• Research is needed to identify appropriate values for individuals or groups and customize AI while maintaining ethical principles.Some values may be widely expected, while others may be undesirable in particular contexts.Source: §10.2 · p. 15
• The paper calls for distinct methods to evaluate AI values and calibrate human expectations, including mechanistic interpretability for assessing AI honesty.Source: §10.2 · p. 15

11 Interaction Techniques for Specifying Human Values

• Common feedback styles in NLP/ML are not often studied in HCI, while diverse HCI feedback is not fully used in NLP/ML AI development.Source: §11 · p. 15

12 Challenges in Achieving Alignment

• Outer alignment requires specifying system purposes, while inner alignment requires systems to robustly adopt those specifications; fully specifying human goals remains difficult.Source: §12 · p. 16
• Incomplete objectives can lead designers to use proxy goals such as overseer approval, leaving systems able to exploit loopholes for unintended outcomes.Black-box models can also make apparently correct decisions for reasons that are difficult to discern.Source: §12 · p. 16
• As AI behavior grows more complex, human evaluation may be slow or infeasible, motivating research into reducing supervision effort and assisting supervisors.Source: §12 · p. 16
• Because human values and preferences change, the paper calls for ongoing, mutual alignment that considers long-term cognitive and social effects and draws on cross-domain expertise.Source: §12 · p. 16
• Some researchers hypothesize that advanced AI could seek power after deployment and pose existential risks if it is not properly aligned with human values.Source: §12 · p. 16