Source-linked AI summary
ATLAS: Discovering Agent Strategies through LLM-Guided Abstraction and Automata Learning
Ignacio D. Lopez-Miguel, Andreas Happe, Jürgen Cito, Ezio Bartocci, Bettina Könighofer, Martin Tappler
TL;DR
Existing evaluations of LLM agents provide limited insight into the agents’ strategies, while explicit behavioral models are difficult to construct manually. ATLAS combines LLM-guided trace abstraction with automata learning to recover compact symbolic models, which expose high-level penetration-testing strategies and support analysis, explanation, and symbolic knowledge transfer.
Problem
Research lacks explicit behavioral models of agents, limiting model-based analysis, comparison, monitoring, and validation while manual modeling is impractical.
Method
ATLAS combines LLM-driven trace abstraction with formal automata learning to recover compact probabilistic behavioral models from agent trajectories.
Results
Recovered models exposed high-level penetration-testing strategies and enabled symbolic transfer from a 284B-parameter frontier model to a 14B-parameter model solving 50% of benchmark tasks.
Takeaways & Limitations
ATLAS shows that agent trajectories can serve as inputs for compact symbolic models supporting explanation, comparative analysis, monitoring, auditing, and knowledge transfer.
Takeaways & Limitations
Explanation extraction may collapse important steps into strongly connected components, obscuring their role in successful behavior.
Abstract
from arXiv · showhide
Large Language Model (LLM)-based agents are increasingly used for complex tasks such as software testing and cybersecurity assessment. While these agents demonstrate impressive capabilities, their behavior is difficult to understand, explain, and analyze. Existing evaluations focus mainly on task success and execution traces, offering limited insight into the strategies employed by the agent. We present ATLAS (Automata Learning for Agent Trajectory Analysis and Strategy Discovery), an approach for recovering interpretable behavioral models from agent trajectories. ATLAS combines trace abstraction with automata learning to infer finite-state models that capture observed agent-environment interaction strategies. These models provide human-interpretable insights and support automated analyses of recurring behaviors, decision points, successful task-completion paths, and failure loops. As a proof of concept, we apply ATLAS to trajectories generated by an LLM-based penetration-testing agent. The resulting models expose high-level behavioral strategies for exploiting vulnerable machines that are difficult to identify from raw execution traces alone. We discuss how learned behavioral models can support explainability, model-guided exploration, auditing, and analysis of agentic systems. We further demonstrate symbolic model-based knowledge transfer from powerful frontier models to compact language models. In addition, we show how model transformations can derive concise explanations of agent behavior in a penetration-testing case study comprising 12 vulnerable machines. ATLAS highlights a new opportunity for model-driven engineering: transforming agent trajectories into explicit behavioral models that enable systematic understanding and analysis of otherwise opaque AI agents.
1 Introduction
ATLAS addresses the lack of explicit, interpretable behavioral models for LLM agents by combining LLM-guided trajectory abstraction with stochastic automata learning. The resulting models support analysis tasks including explanation, monitoring, auditing, symbolic knowledge transfer, and formal analysis, with penetration testing as a proof-of-concept domain.
- Approach: ATLAS recovers behavioral models of agentic AI systems from observed interaction traces by automatically learning labelled Markov chains for task-solving behavior.Labelled Markov chains represent sequential stochastic processes, including the uncertainty and statefulness of foundation-model-backed agents.
- Approach: The pipeline collects traces, uses foundation models to abstract noisy actions and observations, and applies automata learning to infer models from the resulting abstractions.Abstract states are labeled by higher-level action-observation categories, while automata learning reconstructs behavioral structure shared across executions.
- Applications: The learned models become engineering artifacts supporting explanation, runtime monitoring, auditing, model slicing, symbolic knowledge transfer, and formal analysis.The paper presents recovered models as enabling model-based analysis, comparison, monitoring, and validation that are difficult without explicit agent models.
- Contributions: The paper’s contributions are an LLM-guided abstraction pipeline, an automata-learning method for behavioral-model recovery, and a proof-of-concept evaluation on penetration-testing agents.The evaluation demonstrates explanation and symbolic knowledge transfer as applications of the recovered models.
- Novelty and motivation: The authors identify their novelty as combining semantic LLM-based abstraction with automata learning to recover AI-agent behavioral models from execution trajectories.They illustrate ATLAS through agentic penetration testing, chosen as a high-stakes domain relevant to concerns about frontier language models’ cyber-offensive capabilities.
2 Learning a Penetration Testing Strategy
ATLAS recovers interpretable penetration-testing strategies by abstracting agent–environment trajectories and learning Markov chains from them. In the Docker Scenario 3 example, the model exposes reconnaissance patterns, recurrent failures, and probabilistic paths to privilege escalation.
- Strategy learning: ATLAS executes HackingBuddyGPT with different LLMs on vulnerable-machine benchmarks and learns symbolic models from recorded trajectories.The models capture both the agent’s behavioral strategy and corresponding system responses.
- Automata learning: Alergia learns a Markov chain from 20 abstract interaction sequences, labeling states with abstract action–observation pairs and edges with probabilities.The resulting chain summarizes behavioral strategies together with system responses.
- Trace abstraction: The abstraction maps actions to categories with optional targets and observations to categories describing output usefulness and affected system entities.This sacrifices low-level detail to emphasize recurring behavioral patterns.
- Learned strategy: The learned model shows that the agent usually begins with reconnaissance by querying user information or checking sudo privileges via sudo -l.After user discovery, it may produce a malformed command or proceed to sudo -l, which can reveal exploitable privileges.
- Learned strategy: 30% of cases succeed immediately after EXPLOITABLE_SUDO when the agent executes sudo -i.In other cases, the agent performs additional exploratory actions before escalating privileges.
- Learned strategy: 0.4 probability links sudo -l to success within two steps, while all observed execution paths eventually reach successful privilege escalation.The model therefore identifies the sudo -l response as likely important for the agent’s decisions.
3 LLM-Guided Abstraction & Model Learning
ATLAS converts raw agent-environment traces into a compact, semantically meaningful symbolic alphabet through LLM-guided abstraction, then applies automata learning to infer probabilistic behavioral models. The approach supports adaptive abstraction while making additional trace assignment relatively inexpensive with smaller language models.
- Overview: ATLAS has two components: LLM-driven trace abstraction and automata learning over the resulting symbolic alphabet.The abstraction translates raw interaction traces into symbols, while model learning infers probabilistic finite-state behavioral models.
- Abstraction: LLM-guided abstraction replaces concrete commands and observations with semantically meaningful categories that preserve behavioral intent for automata learning.Actions map to categories with optional targets, while observations use Value_Target labels such as DISCOVERY_SUID, EXPLOITABLE_FILE, and ERROR_NONE.
- Abstraction: The three-stage abstraction pipeline categorizes interactions, normalizes redundant labels across runs, and assigns normalized action-output pairs to each trace.A retry mechanism with format-correction prompts handles malformed LLM outputs during per-trace assignment.
- Abstraction: Unlike manual coarse abstractions or feature engineering, LLM categorization adapts to each interaction’s semantic content, environment, and task to produce a stable symbolic alphabet.The resulting alphabet bridges unstructured agent behavior and formal model inference.
- Model learning: Alergia merges abstracted interaction trees into probabilistic Markov chains over action-observation pairs, capturing agent behavior and environmental responses.Automata learning can infer latent Markov states that encode relevant history from partial observations.
- Cost of Abstraction: Category discovery requires a powerful LLM, whereas abstraction assignment is a classification task that smaller language models can perform for relatively cheap trace refinement.This cost distinction is illustrated by the dynamic mode described in Section 4.
4 Case Study: Applications of Learned Behavioral Models
ATLAS is applied to 12 vulnerable Linux virtual machines to learn behavioral models and demonstrate symbolic knowledge transfer to compact language models and concise explanations of successful strategies. Dynamic transfer is most effective, while model transformations substantially reduce model size but may collapse important steps.
- ATLAS Setup: ATLAS learns Markov chain models from 20 penetration-testing runs per virtual machine, using abstracted traces labeled with success or failure.The experiments use DeepSeek V4 Flash in non-reasoning mode with a 20-turn limit and exclude Scenario 4 because the SSH setup cannot support interactive shell commands.
- Knowledge Transfer: Dynamic mode is most effective because it tracks abstract model states and suggests one action at a time, separating symbolic planning from language-model execution.After each command, a classifier maps the concrete command-observation pair to abstract categories so the learned model can advance its state; anti-repetition and deadend detection are included.
- Knowledge Transfer: Common Path can reduce success for small models because its relatively large hints are difficult to process, whereas symbolic knowledge transfer remains possible without replicating every successful behavior.The same effectiveness pattern is observed for ministral-8b, and the recovered models enable transfer from a frontier model.
- Explanation Models: Observation-focused model transformations remove reconnaissance details to make learned strategies easier to interpret and produce concise explanation models.For Scenario 13, the transformation reduces the model from 41 to 7 states, but important steps may be collapsed into strongly connected components.
5 Conclusion
ATLAS combines LLM-driven abstraction with formal automata learning to recover compact symbolic models of agent behavior from trajectories. The work frames these learned automata as engineering artifacts and outlines applications in comparative analysis, assurance, monitoring, auditing, and future model-management research.
- Conclusion: ATLAS combines LLM-driven abstraction with formal automata learning to recover probabilistic behavioral models from agent trajectories.The framework transforms trajectories into compact symbolic behavioral models and treats trajectories as first-class modeling inputs.
- Conclusion: The work presents a conceptual framing, initial pipeline, and case-study demonstration connecting model recovery, runtime analysis, and AI-based systems.It positions the contribution as opening a broader research agenda rather than as a completed solution.
- Conclusion: Learned automata can support comparative analysis across agent releases, model-based assurance, safety and security monitoring, and auditing.The conclusion treats behavioral models as engineering artifacts for systematic analysis of increasingly autonomous agents.
- Conclusion: Future work should investigate validation methods, abstraction strategies, and model-management techniques for behavioral models of increasingly autonomous AI agents.The authors invite further study of these directions within the MODELS community.
6 Future Plans
ATLAS has empirically recovered behavioral models from penetration-testing agent trajectories, while future work will test its generality, scalability, and broader model-driven uses. The authors also identify open questions about model usefulness and envision applications in monitoring, planning, policy recovery, and model management.
- ATLAS empirically recovers behavioral models from agentic AI trajectories in penetration-testing applications.
- Demonstrating Generality: Future work will recover behavioral models in domains beyond penetration testing, including web navigation and agentic software testing, and evaluate their usefulness to AI-agent users.
- Model-Driven Engineering: Planned model-driven activities include high-level planning from failure patterns and symbolic knowledge deduction to adapt agentic behavior.
- Scalability: Long trajectories, changing tasks, and heterogeneous environments may challenge abstraction discovery and model learning, requiring useful behavioral-model requirements.An open question is whether models should offer coarse, complete coverage or partial, fine-grained detail.
- Closing: Combining model-driven engineering, formal methods, automata learning, machine learning, and symbolic AI could enable runtime monitoring, symbolic planning, policy recovery, and model management.
Ethics and Privacy Statement
The case study’s use of LLMs for penetration testing raises concerns about potential misuse by malicious actors. The authors refer readers to the original HackingBuddyGPT publication for a fuller discussion of ethical concerns.
- Ethics and Privacy Statement: LLM-based penetration testing raises concerns about potential application by malicious actors.The authors identify misuse risk as an ethical concern associated with the case study.
- Ethics and Privacy Statement: The paper directs readers to HackingBuddyGPT [9] for a more thorough discussion of ethical concerns.This reference is provided because the work builds upon existing research in the area.