Source-linked AI summary
Avatar Intelligence: Amplifying Human Agency Through Persistent Personal Agents
Zheng Wu, Xinbei Ma, Yao Yao, Zhaoyuan Huang, Haowen Hu, Yijie Lu, Taolei Wang, Yifei Yang, Wuyue Xiong, Pengzhou Cheng, Zhuosheng Zhang
TL;DR
Personal agents can handle long-horizon tasks, but users still spend effort specifying goals, resolving ambiguity, and verifying results. The paper advocates persistent agents that represent users across contexts, and frames their design and evaluation around delegated decisions, human oversight, and user control. It concludes that progress should be shown through decisions users can reliably delegate and the effort needed to obtain outcomes they endorse.
Problem
Personal agents can execute long-horizon tasks, yet users still bear substantial effort specifying goals, resolving ambiguity, and verifying outcomes.
Source: §1 · pp. 3–4Method
The paper frames Avatar Intelligence as persistent user representation and sustained decision delegation, with a maturity roadmap and an oversight-adjusted evaluation index.
Source: §1 · p. 4Results
The paper identifies a research agenda of cross-context user representations, user-situated rehearsal, and delegation supported by technical control, accountability, and recourse.
Source: §5 · p. 13Takeaways & Limitations
Progress should be demonstrated through the decisions users can reliably delegate and the effort required to obtain outcomes they endorse.
Source: §5 · p. 13The roadmap’s autonomy applies within delegated domains, and users retain authority to revise goals and revoke delegation at every level.
Source: §2.2 · p. 6
Abstract
from arXiv · showhide
Personal AI agents are increasingly capable of handling long-horizon tasks across applications, yet users often need to invest substantial effort in specifying goals, resolving ambiguity, and verifying outcomes. We argue that progress should be measured by how effectively agents amplify human agency: enabling people to pursue their goals with less routine supervision while retaining control over consequential decisions. We advocate Avatar Intelligence, a paradigm of persistent personal agents that represent users' goals, values, and constraints through sustained decision delegation across contexts and time. We describe a maturity roadmap from human-directed execution to autonomous operation within delegated domains, and introduce the Agentic Oversight Index (AOI) as a framework for evaluating delivered value against agent-side operational cost and human oversight burden. We identify three core research challenges: identity-continuous personalization that carries user preferences across tasks, user-situated evolution through Rehearsal Engineering, and safety mechanisms that combine technical control with accountability and recourse. These directions define a research agenda for reliable decision delegation to personal agents across tasks and over time.
1 Introduction
Personal agents can execute long-horizon tasks, but users still carry substantial specification, coordination, and verification work. Avatar Intelligence frames progress as reducing routine supervision while preserving users’ control over goals and consequential decisions.
1 Introduction
- Users still resolve ambiguity, correct interpretations, and judge sufficiency even when agents retrieve and summarize documents.Execution effort can fall while specification, coordination, and verification burdens remain substantial.Source: §1 · pp. 3–4
- Avatar Intelligence describes persistent personal agents representing users’ goals, values, and constraints across tasks and over time.They should maintain commitments, adapt to changing circumstances, and know when to seek clarification or authorization.Source: §1 · p. 4
- The paper characterizes Avatar Intelligence through persistent user representation and sustained decision delegation, supported by a user-centric system architecture.Source: §1 · p. 4
- The proposed maturity roadmap spans L0–L3, transferring operational and decision responsibilities within defined delegation scopes.Source: §1 · p. 4
- The Agentic Oversight Index accounts for delivered value, agent-side operational cost, and human oversight burden.Source: §1 · p. 4
- The paper identifies identity-continuous personalization, user-situated evolution through rehearsal, and safety mechanisms for controlled, accountable delegation as core research directions.Source: §1 · p. 4
2 Avatar Intelligence
Avatar Intelligence treats persistent representation of a user as the basis for delegated action across contexts and time. The section develops this idea through a user-centric architecture, a domain-scoped autonomy roadmap, and an index that accounts for human oversight.
2 Avatar Intelligence
- Avatar Intelligence frames agents as persistent digital representatives embodying user identity, preferences, and values, rather than tools for isolated tasks.Source: §2 · p. 4
2.1 Avatar as a User-Centric System
- The user-centric system combines a persistent memory substrate with perception, planning, decision, and execution in a closed loop.The figure depicts the user model and memory supporting this inner loop.Source: Figure 2 · p. 5, §2.1 · p. 5
- Task-specialized work, assistant, social, and custom clones share one user model and connect to heterogeneous tools, applications, environments, and agents.The clone layer supports work continuity, proactive service, and capability expansion.Source: §2.1 · p. 5
2.2 Roadmap
- The L0–L3 roadmap moves from user-involved subprocess execution through longer workflows and checkpointed decisions to end-to-end autonomy in delegated domains.L2 assigns routine decisions and orchestration to the avatar with calibrated checkpoints; L3 shifts humans primarily to post-hoc auditing, dispute resolution, and policy updates.Source: §2.2 · p. 6
- Autonomy is scoped to delegated domains: the same avatar may operate at different levels across domains, while users can revise goals or revoke delegation at every level.Source: §2.2 · p. 6
- The roadmap’s research transition is from improving execution quality to engineering bounded-risk decision delegation and institutional-grade assurance for high-stakes autonomy.Source: §2.2 · p. 6
2.3 Agentic Oversight Index (AOI)
- The Agentic Oversight Index is an oversight-adjusted net-value score: AOI(A, τ) = U(A, τ) − λP(A, τ), with U as normalized net value and P as normalized oversight burden.U reflects delivered user value after agent-side operational cost; λ weights the penalty for human oversight.Source: §2.3 · pp. 6–7
- The index always penalizes additional oversight at fixed net value, and alternative operationalizations of U and P can follow the same principle.The score has no finite lower bound so arbitrarily large human workload is not treated as merely equally bad.Source: §2.3 · p. 7
3 Applications
Current products carry work across tasks and seek user approval for consequential actions, marking a shift toward oversight within delegated tasks. The application scenarios show how avatars can preserve commitments, respect role-specific authority and risk preferences, and coordinate across users.
3.1 Current Industry Products
- Current products carry work forward across tasks: Grok Bot preserves context, Muse pursues ongoing goals, and dots returns decisions requiring human judgment to users.These systems mark a transition from human-in-the-loop execution toward human-on-the-loop oversight within delegated tasks.Source: §3.1 · pp. 7–8
3.2 Application Scenarios
- In personal-life administration, an avatar can seek a later departure before returning an immovable conflict to the user, protecting commitments a task-by-task assistant might overlook.The example is a work trip conflicting with a previously promised parent–teacher meeting.Source: §3.2 · pp. 8–9
- For organizational work, an avatar can postpone an internal launch and notify teams, but cannot waive an engineering blocker or revise another team’s customer commitment.Coordination depends on shared project information and the distinct authority of the roles involved [22].Source: §3.2 · p. 9
- For financial decisions, an avatar should pause a purchase that breaches the user’s exposure limit and seek a decision, even if the initial instruction was to buy.The user's risk preferences shape actions, while changes to those preferences remain under user control.Source: §3.2 · p. 9
- When coordinating across avatars, agents can exchange acceptable meeting windows and settle on a time that respects both users’ constraints, or return the choice to users.Negotiation should avoid exposing more of either user's preferences than necessary.Source: §3.2 · p. 9
4 Open Challenges
Avatar Intelligence faces challenges in learning user decision patterns, evolving within real environments, and keeping persistent delegation safe and controllable. The paper proposes identity-continuous personalization, user-situated rehearsal, and technical controls with financial recourse as research directions.
4 Open Challenges
- The agenda is to learn users’ cognitive and decision patterns, evolve in real-world environments, and keep autonomy safe and controllable while preserving proactive assistance.These are framed as challenges at the interface between users and advancing agent technology.Source: §4 · p. 9
4.1 Personalization: From Behavioral Alignment to Identity-Continuous Avatars
- Memory-heavy approaches can be brittle because coverage is incomplete and preferences are context-dependent; identity-continuous models aim to generalize across tasks while remaining stable yet updateable.The proposed user model captures stable decision structure and transfers across tasks while evolving without drifting from users’ values and decision structure.Source: §4.1 · pp. 10–11
4.2 Evolving with Specialization
- Rehearsal Engineering specializes agents to users’ real environments through traces and preferences, staged scenarios, oversight-based evaluation, and bounded updates to memory, skills, and policy.The comparison distinguishes rehearsal's staged scenarios and user-situated feedback across episodes from prompt, context, and harness engineering; situated rehearsal may help internalize user-specific feedback while remaining auditable and corrigible.Source: Table 1 · p. 11, §4.2 · p. 12
4.3 Safety
- Safety combines continuous technical controls that limit failures and consequences with Agent Insurance that allocates residual financial risk and compensates covered losses, without guaranteeing correct or user-aligned execution.The paper presents these layers as infrastructure for delegation across capability levels; both technical control and defined recourse remain necessary.Source: §4.3 · pp. 12–13
5 Conclusion
Avatar Intelligence frames personal agents as continuing delegates that help users pursue goals across tasks and time with less routine supervision and control over consequential choices. It outlines system design, a delegation roadmap, and an evaluation framework, then identifies research directions and a criterion for progress.
5 Conclusion
- Avatar Intelligence centers on continuing decision delegation, aiming to help users pursue goals across tasks and time with less routine supervision and control over consequential choices.Source: §5 · p. 13
- The paper outlines a user-centric architecture, a delegation roadmap, and the Agentic Oversight Index to connect the goal of agency amplification with system design and evaluation.Source: §5 · p. 13
- Its research agenda combines cross-context user representations, user-situated rehearsal, and safety with accountability and recourse; progress should be measured by reliably delegable decisions and effort for endorsed outcomes.Source: §5 · p. 13
A Formalization of Avatar Personalization
The formalization treats an avatar’s identity representation as a computable proxy for user cognition, separating stable personality structure from stored user-specific knowledge and experience. It frames personalization as recovering that representation from behavior while maintaining cross-context consistency and stability as the user and avatar co-evolve.
Identity representation
- The identity representation combines relatively stable personality structure, such as values and risk preference, with a structured store of user-specific knowledge and experience.This separates how a user thinks from what the user knows.Source: A Formalization of Avatar… · p. 16
Personality mining as inverse cognition modeling
- Personality mining is framed as recovering latent structure from observed behavior so behavior can be predicted across contexts, rather than matched locally.Source: A Formalization of Avatar… · p. 16
Cross-context consistency as a predictability constraint
- The same identity should induce consistent decision structure across heterogeneous tasks, preventing behavior from degenerating into context-local optimization.Source: A Formalization of Avatar… · p. 16
Co-evolution and stability
- Because the user and avatar adapt over time, the formalization imposes a stability condition to prevent drift, collapse, or unintended identity substitution.The learned identity is required to remain a faithful and stable proxy of the user.Source: A Formalization of Avatar… · p. 16
B One Instantiation of AOI
This AOI instantiation distinguishes delivered user value, agent-side operational cost, and human oversight burden. It normalizes net value and penalizes normalized oversight, with illustrative forms for value and cost components.
AOI
- The AOI uses normalized net value and a monotonic penalty for human oversight, with oversight counted once rather than also included in agent-side operational cost.The appendix presents its functional forms as illustrative; its stated requirements are normalized net value and a single oversight penalty.Source: B One Instantiation of AOI · pp. 16–17
Human oversight burden
- Human oversight burden aggregates intervention frequency, supervision time, decision complexity, and accountability or audit load, weighted for the evaluation setting.It is normalized against the effort required when a human performs the task without an avatar; P = 0 indicates no oversight, P = 1 human-only effort, and P > 1 greater workload.Source: B One Instantiation of AOI · p. 17
Delivered value
- Delivered value comprises task value and optional experiential value, with task value modeled using a saturation form.The optional experiential term can capture satisfaction and calibrated trust without conflating them with oversight.Source: B One Instantiation of AOI · p. 17
Operational cost
- Agent-side operational cost comprises fixed execution consumption and learning cost, for which the appendix records one possible parameterization.Source: B One Instantiation of AOI · p. 17