Source-linked AI summary
Agents in the Large: Perception-Centered Architecture for Persistent Agents
Shihan Dou, Haoxiang Jia, Shichun Liu, Feng Chen, Chenhao Huang, Yujiong Shen, Shaofan Liu, Jiayi Chen, Jiahang Lin, Honglin Guo, Qianyu He, Minghao Guo, Ziyi Ye, Pluto Zhou, Tao Gui, Qi Zhang, Xuanjing Huang
TL;DR
Language agents remain largely organized around bounded user tasks, while persistent assistance requires service across long-lived settings whose needs, context, and procedures change. The paper proposes Pera, which centers continual perception and lifecycle control to construct tasks that maintain and adapt ongoing service. Pera organizes prior work, presents a case study, and identifies actionable research directions, while realistic scalable benchmarks for persistent service remain underdeveloped.
Problem
Existing language-agent frameworks largely target bounded user-specified tasks, leaving no architectural account of persistent assistance in changing long-lived settings.
Method
Pera organizes persistent agents around perception and control components that convert external and internal service-relevant signals into lifecycle tasks.
Results
Pera retrospectively organizes recent agent work, illustrates persistent adaptation through a model-development case study, and offers actionable directions for future research.
Takeaways & Limitations
Persistent agents require broader perceptual scope and lifecycle-level work that can adapt ongoing assistance across changing settings and future episodic tasks.
Takeaways & Limitations
Realistic and scalable benchmarks for evaluating persistent service across evolving goals, accumulated context, diverse tasks, and self-adaptation remain underdeveloped.
Abstract
from arXiv · showhide
Cognitive language agents have achieved substantial progress by equipping language models with memory, tools, and decision-making procedures, enabling agents to reason and act in interactive environments. Existing frameworks largely cast these agents as systems for solving user-specified, bounded tasks. An increasingly important goal is for language agents to provide persistent assistance in long-lived settings where user needs, context, and service procedures persist and change, and to remain useful across the broad range of tasks that arise over time. Yet we still lack a framework to characterize persistent AI agents, organize existing work, and guide future development. To this end, we propose a Perception-Centered Architecture for Persistent Agents (Pera). Pera describes a persistent agent organized around perception and control components that continually perceive service-relevant signals from episodic task executions, internal context, and changes in the surrounding environment, and use these signals to construct lifecycle tasks. These tasks drive the ongoing operation and adaptation of the agent's service procedures. We use Pera to retrospectively organize recent work, examine a detailed case study, and offer forward-looking insights for building more capable persistent agents. Just as software engineering moved from programming in the small to programming in the large, Pera frames the evolution of language agents as an analogous architectural transition toward long-lived, adaptive intelligence systems.
1 Introduction
Language agents are shifting from bounded task completion toward persistent assistance in long-lived settings. Pera addresses this gap with perception and control components that turn ongoing changes into lifecycle tasks for adaptation.
- Motivation: Language agents are increasingly expected to provide persistent assistance beyond completing individual user-specified tasks.This extends their operational scope to long-lived settings where needs, context, and procedures persist and change.
- Motivation: Most existing language agents remain cognitive systems organized around a current user task, coordinating models, memory, tools, and decision-making procedures.Their objectives and evaluations typically center on bounded task execution, even when individual tasks run continuously for extended periods.
- Motivation: Persistent agents must continually perceive changes in the user, environment, internal context, and service procedures, including needs that have not been explicitly articulated.Active perception enables agents to recognize signals relevant to future service rather than waiting for those signals to reappear through a later task.
- Pera: Pera augments cognitive agents with perception and control components that sense service-relevant signals and formulate lifecycle tasks.Signals come from environmental changes, task execution, and internal context; lifecycle tasks can update context, revise procedures, or prepare support for emerging needs.
- Pera: Pera distinguishes episodic tasks that complete specific work from lifecycle tasks that sustain and improve continuing service.The paper uses this distinction to organize prior work, analyze a detailed case study, and suggest directions for future persistent agents.
2 Background
The background motivates a shift from controlling individual task computations to controlling relations, information acquisition, and persistent service across broader architectural scopes. Active perception supplies the feedback relation through which activities guide sensing and observations update subsequent control.
- 2.1 Programming in the small and programming in the large: Programming in the small concerns computation within individual units, whereas programming in the large organizes relations among units and system-level properties.Architectural control is needed when interfaces, dependencies, or interactions create properties that no individual module addresses alone.
- 2.2 Language agents as agents in the small: Cognitive language-agent architectures combine models, memory, actions, and decision procedures to execute bounded episodic tasks.An episodic task is organized around an objective that determines relevant information and when execution is complete.
- 2.2 Language agents as agents in the small: An agent remains an agent in the small when an episodic task is its highest-level explicitly organized unit, even with long execution or persistent state.Planning, tool use, testing, long-term memory, and reusable procedures do not enlarge architectural scope when subordinate to the active episodic objective.
- 2.3 Active perception in adaptive intelligent systems: In dynamic environments, perception must selectively acquire information because resource-bounded agents cannot process all available information continuously.Adaptive control guides which environmental information enters reasoning and at what level of detail.
- 2.3 Active perception in adaptive intelligent systems: Active perception couples information acquisition to an agent’s activities and information needs through feedback between observations, assessment, reasoning, and perceptual choices.What is sensed, when it is sensed, and its level of detail can change with the activities and conditions being served.
3 From agents in the small to agents in the large
Agents in the large extend control beyond individual episodic objectives to the continuing service of a long-lived setting. Active perception makes cross-task changes observable, while lifecycle tasks turn selected changes into bounded interventions that maintain or improve future service.
- 3.1 Episodic tasks within long-lived settings: A long-lived setting spans multiple objectives and includes persistent users, artifacts, constraints, decisions, and procedures whose effects survive individual tasks.It is the scope of continuing service, not simply a particular memory representation.
- 3.1 Episodic tasks within long-lived settings: Unlike long-horizon tasks, persistent agents coordinate conditions across task boundaries because earlier decisions, delayed feedback, artifacts, policies, tools, and needs can change over time.Previously valid context may become stale, and once-successful procedures may no longer fit current conditions.
- 3.2 Active perception makes the setting observable: Active perception extends information acquisition beyond the current task by inspecting completed-task outcomes, persistent context, reusable procedures, and environmental changes.Its sensing adapts to what the agent currently needs to know, and evidence across tasks can reveal persistent mismatches.
- 3.2 Active perception makes the setting observable: Perception alone does not determine a response: control must distinguish local exceptions, persistent changes, and conditions outside the agent’s authority before selecting an intervention.This control level evaluates perceived conditions and turns selected changes into bounded interventions.
- 3.3 Lifecycle tasks make perceived changes actionable: Lifecycle tasks maintain or improve conditions shared by future tasks, completing when required evidence or persistent change has been established rather than when a one-off user outcome is produced.They can assess changes, update stale context, revise unreliable procedures, verify adaptations, or prepare for recurring needs.
- 3.3 Lifecycle tasks make perceived changes actionable: Task purpose and completion criterion, not proactivity or initiator, determine whether work is episodic or lifecycle-level.Preparing one report is episodic, whereas correcting a preference intended to govern future reports is lifecycle-level.
- 3 From agents in the small to agents in the large: Pera adds organization above episodic execution: perception observes cross-task change, lifecycle control selects interventions, and task cognition executes the resulting bounded tasks.This constitutes the transition from agents in the small to agents in the large.
4 Pera: A Perception-Centered Architecture for Persistent Agents
Pera extends cognitive language agents with lifecycle perception and control so they can sustain service across changing long-lived settings. It distinguishes episodic work from lifecycle work and connects signals, task execution, review, and persistent adaptation in a recurring loop.
- 4 Pera: A Perception-Centered Architecture for Persistent Agents: Pera adds lifecycle perception and a control plane to cognitive language agents, extending them from episodic execution to continuing service.Lifecycle perception interprets external and internal signals, while the control plane decides whether further evidence or work is needed.
- 4 Pera: A Perception-Centered Architecture for Persistent Agents: Episodic tasks pursue concrete user-facing outcomes, whereas lifecycle tasks establish or restore persistent conditions needed for future service.Both use the same task-level machinery, but lifecycle tasks additionally return evidence for lifecycle-level review.
- 4.5 Lifecycle-Level Decision-Making: Pera’s recurring loop makes long-lived changes observable, turns relevant changes into executable lifecycle work, and feeds execution outcomes back into perception and control.Its action space includes sensing, dispatching, instantiation, reasoning, retrieval, updating, and grounding.
- 4.1 Signals and Perception: Lifecycle perception captures signals from external environments, task executions, internal context, and agent components, then interprets them for lifecycle-level judgment.Signals may represent individual events or conditions inferred by aggregating related events.
- 4.1 Signals and Perception: Sensors detect signals and signal processors interpret them, but choosing sensing intensity, granularity, and aggregation remains a core design challenge.Relevant conditions may require sustained observation across events or state changes, while many external and internal signals may be irrelevant.
- 4.2 Lifecycle Tasks and 4.4 Task-Level Decision-Making: The control plane converts interpreted signals into lifecycle task packages specifying objectives, procedures, context acquisition, failure recovery, and review.Task instantiation applies these packages to the agent specification, creating runtimes that repeatedly plan and act until completion or failure.
5 Case Study: Persistent Reliability for an Evolving Model-Development Workload
The case study models four dependent workloads on a shared cluster to show how persistent context and evolving requirements can expose a mismatch in reliability service. Pera converts that mismatch into a lifecycle task that updates shared procedures for later workloads.
- 5.1 A Long-Lived Model-Development Setting: Four successive workloads—dense training, inference and evaluation, data processing, and MoE training—share persistent artifacts, operating conditions, and service procedures.Each workload is an episodic task, but their dependencies make them part of one long-lived setting.
- 5.1 A Long-Lived Model-Development Setting: The transition toward MoE training exposes a mismatch between the evolving workload and reliability procedures still based on the earlier regime.Accumulated context, the upcoming configuration, and resource plans reveal the mismatch before the next episodic task begins.
- 5.2 Architectural Comparison: The comparison evaluates whether architectures preserve cross-task information, perceive changes beyond current objectives, and turn those changes into work maintaining future service procedures.Table 1 organizes representative systems and Pera by perception, control, and context.
- 5.2 Architectural Comparison: Plan-and-Act controls each active task, while A-MEM preserves episodic experience without turning changing workload requirements into maintenance work for the reliability service.PaLM-E expands observations within the current task, whereas ContextAgent proactively initiates episodic assistance without maintaining shared sensing and diagnostic procedures.
- 5.2 Architectural Comparison: Pera combines accumulated context with external and internal signals to detect the emerging mismatch and dispatch a lifecycle task revising shared sensing, processing, and diagnostic procedures.The revised procedures become part of continuing service state and support subsequent workloads while preserving earlier-stage support.
- 5.2 Architectural Comparison: After validation, the subsequent MoE training remains an episodic task served by reliability procedures adapted to the evolving project.This connects cross-task change detection with persistent adaptations that benefit future episodic tasks.
6 Actionable Insights
Pera’s actionable directions focus on making persistent agents better at sensing, representing, learning from, and acting on evolving context across tasks. They also emphasize lifecycle control, safe self-improvement, and evaluation through realistic long-lived service.
- Adaptive sensing: Persistent agents need adaptive sensing that covers human, physical, digital, and internal signals while allocating monitoring effort selectively.Broad sensing expands relevant coverage, whereas selective sensing reduces unnecessary computational expense by adjusting sources, timing, frequency, and granularity.
- Adaptive sensing: Evolving sensors should revise software detectors, add data sources, and recalibrate, repair, replace, or introduce physical sensors as settings change.
- Signal representation: Structured signal representations and cross-signal aggregation can make changing conditions comparable over time and reveal higher-level problems from distributed evidence.Repeated failures across tasks can indicate that a previously effective procedure has become obsolete, whereas a single failure may not establish a persistent problem.
- Context learning: Persistent agents must continually learn unfamiliar knowledge from messy context and track when that knowledge remains valid as circumstances change.Examples include unfamiliar workflows, revised policies, changing projects, and corrections from new evidence.
- Proactivity: Proactivity requires discovering emerging needs from accumulated context, deciding whether to act, and calibrating initiative to confidence, benefit, and risk.Possible responses range from continued observation or background preparation to more active intervention.
- Lifecycle control: Lifecycle control must coordinate episodic and lifecycle tasks that differ in urgency, dependencies, resource demands, and effects on persistent components.Safe self-improvement further requires risk- and reversibility-sensitive validation, while realistic benchmarks should evaluate sustained adaptation across evolving long-lived service.
7 Discussion
The discussion positions Pera among continual-learning, agent-framework, and perception-centered traditions while identifying a gap in architectures for persistent service. It argues that persistent agents need broader proactive perception and lifecycle-oriented control than current task-centered systems provide.
- Continual and lifelong learning: Continual-learning formulations retain experience from completed task streams, but persistent agents must also perceive service-relevant external and internal changes that are not explicit tasks.
- Continual and lifelong learning: Persistent agents must operate prospectively by uncovering new contextual information as settings evolve and using it to adapt ongoing assistance.
- Frameworks for agents: Existing work either develops specific lifelong-learning approaches or surveys agent capabilities, leaving no unified conceptual architecture for persistent-agent operating principles.Pera addresses this gap by proposing a perception-centered architecture and using it to organize recent work.
- Perception-centered architectures: Earlier perception-centered architectures regulate behavior for current tasks, whereas Pera treats perception as supporting emerging-need identification and service evolution across a setting’s lifetime.
- Perception-centered architectures: Pera characterizes how external and internal signals generate lifecycle tasks, how lifecycle and episodic tasks interact, and how their interaction sustains and adapts service over time.
8 Conclusion
Pera is presented as a perception-centered architecture for persistent agents and a foundation for developing proactive, long-lived, adaptive intelligent systems.
- The paper presents Pera as a perception-centered architecture for persistent agents.
- Pera is intended to provide a conceptual and architectural foundation for agents in the large.
- The proposed foundation supports developing proactive, long-lived, and adaptive intelligent systems.