PaperlayerView exact revision on arXiv

Source-linked AI summary

DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation

Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami, Andrea Maracani, Ignacio Castro, Matthew Purver, Mete Ozay, Savas Ozkan, Taha Ceritli

arXiv:2606.29961v1cs.LGcs.AI

TL;DR

Capable memory-augmented agents are costly to deploy on resource-constrained devices because they rely on large models and long contexts. DuoMem distils procedural problem-solving into compact models through context- and parameter-space mechanisms, raising a 4B model’s ALFWorld success from 4.3% to 77.9% while making it over 3× faster than a 72B teacher.

  • Problem

    Capable procedural agents require large models and long contexts, while smaller models show weaker procedural reasoning and memory use, limiting on-device deployment.

  • Method

    DuoMem transfers procedural memory from a large teacher to compact students through teacher-generated input memories and LoRA fine-tuning on successful teacher trajectories.

  • Results

    4.3% to 77.9% task success: DuoMem boosts a 4B model on ALFWorld while completing tasks over 3× faster than the 72B teacher.

  • Takeaways & Limitations

    DuoMem makes compact models practical for edge deployment, with complementary distillation components whose benefits vary by the model’s initial capabilities.

  • Takeaways & Limitations

    Evaluation is limited to ALFWorld and a single Qwen2.5-72B-Instruct teacher, leaving other procedural domains and teacher configurations for future study.

Abstract

from arXiv · show

Large Language Model (LLM)-based agents can solve complex procedural tasks by interacting with environments over multiple turns, but this ability typically depends on large models, long contexts, and repeated inference calls. This makes advanced memory-augmented agents difficult to deploy on resource-constrained devices. We introduce DuoMem, a dual-space distillation framework that transfers procedural problem-solving ability from a large teacher model to compact student models. DuoMem distils in two complementary spaces: (1)context-space distillation, which replaces student-generated memories with higher-quality teacher-generated procedural memories prepended to the student's input, and (2)parameter-space distillation, which fine-tunes lightweight LoRA adapters on successful teacher trajectories. Evaluated on ALFWorld, a challenging embodied decision-making benchmark, DuoMem boosts a 4B-parameter model from 4.3% to 77.9% task success rate, closing most of the gap to a 72B teacher model (87.1%), while adding fewer than 10M trainable parameters and only a few megabytes of pre-computed teacher memories. Moreover, the DuoMem-enhanced 4B model completes tasks over 3x faster than the 72B teacher in wall-clock time, making it viable for real-time edge deployment, which would be challenging for the teacher.Extensive ablations across eight models spanning 2B-72B parameters reveal that both distillation axes contribute complementary

1 Introduction

DuoMem addresses the gap between capable but resource-intensive LLM agents and weaker compact models by distilling procedural memory from a large teacher into students through complementary context-space and parameter-space mechanisms. On ALFWorld, it substantially improves a 4B model’s task success while reducing deployment overhead.

  • Motivation: 30B–70B+ parameter models and long multi-turn contexts create memory, energy, and latency barriers to edge deployment.These costs hinder deployment on smartphones, robots, and embedded systems.
  • Motivation: 2B–12B models are attractive for on-device use but show weaker procedural reasoning and often fail to exploit memory effectively.The limitation persists even when smaller models are provided with memory.
  • DuoMem: DuoMem transfers procedural memory from a large teacher to compact students through complementary context-space and parameter-space distillation.The framework operates in two distinct spaces to address the deployment gap.
  • Context-Space Distillation: Teacher-generated procedural memories are extracted offline and prepended to the student’s context without modifying model parameters.Context-Space Distillation replaces student-generated memories with relevant teacher-generated memories at inference.
  • Parameter-Space Distillation: Successful teacher trajectories fine-tune lightweight LoRA modules, teaching students to follow procedural patterns from expert demonstrations.This parameter-space mechanism uses successful training-task trajectories rather than changing the full student model.
  • Evaluation: 4.3% to 77.9% task success lifts a 4B model on ALFWorld, closing 89% of the gap to the 72B teacher while completing tasks over 3× faster.DuoMem adds fewer than 10M parameters and requires only a few megabytes of pre-computed memories.

2 Related Work

Prior work augments LLM agents with procedural memories for multi-step tasks and transfers capabilities from larger teacher models to smaller students through knowledge distillation. Parameter-efficient methods such as LoRA provide the adaptation mechanism used for DuoMem’s parameter-space component.

  • Procedural Memory for LLM Agents: Procedural-memory approaches support multi-step task completion through skill libraries, memory streams, continual accumulation, retrieval-augmented planning, and knowledge-constrained action planning.MemP is identified as most closely related to DuoMem, although the supplied passage ends before describing it further.
  • Knowledge Distillation for LLMs: Knowledge distillation transfers capabilities from a larger teacher to a smaller student model, including through chain-of-thought rationales and specialization for multi-step reasoning.The passage also cites a comprehensive survey of LLM distillation, but its description is truncated.
  • Parameter-Efficient Fine-Tuning: LoRA and QLoRA enable efficient adaptation by training a small number of low-rank parameters while freezing the base model.DuoMem uses LoRA for parameter-space distillation by training on successful teacher trajectories to transfer behavioral patterns.

3 Method

DuoMem combines teacher-generated procedural memories with LoRA fine-tuning on successful teacher trajectories. Its context-space and parameter-space mechanisms improve complementary aspects of student behavior and are applied together at inference.

  • Procedural Memory: Procedural memory extracts concise scripts from completed trajectories and appends retrieved scripts alongside a reference trajectory, environment state, and task description.The input is x = [s; τref; e; d; m1; . . . ; mk], with MemP providing both k scripts and one full trajectory.
  • Context-Space Distillation: Context-space distillation replaces student-generated summaries with teacher-generated memories for the most relevant completed trajectories without modifying student parameters.Teacher memories are generated offline and stored as text files, typically requiring only a few megabytes for the task suite.
  • Parameter-Space Distillation: Parameter-space distillation fine-tunes low-rank LoRA adapters on successful teacher trajectories while keeping the base student model frozen.The teacher retries unsolved training tasks up to 5 times and retains only successful trajectories; each trajectory supplies decision-point samples for action imitation.
  • Parameter-Space Distillation: Training inputs use one of three strategies: Full History, Latest Only, or Last-5, differing in how much recent interaction context supplies the training signal.Full History predicts all assistant turns, Latest Only uses only the most recent turn, and Last-5 provides the five most recent turns.
  • Combined Distillation: DuoMem first applies LoRA fine-tuning and then prepends teacher-generated memories at inference, improving how the student acts and what it sees through complementary mechanisms.The paper states that their combination yields gains substantially exceeding either component alone.

4 Experimental Setup

Experiments evaluate DuoMem on ALFWorld, a text-based embodied environment requiring multi-step planning and interaction, using Gemma and Qwen models spanning 2B–72B parameters. Qwen2.5-72B-Instruct serves as teacher, with diverse oversampled trajectories, LoRA hyperparameter search, and success rate, steps, and runtime as evaluation metrics.

  • Environment: ALFWorld contains six task types requiring multi-step planning and interaction through textual observations, goals, and actions.The environment is derived from ALFRED and includes tasks such as picking up objects, heating or cooling items, and examining objects.
  • Models: Models span 2B–72B parameters across the Gemma and Qwen families, with Qwen2.5-72B-Instruct designated as the teacher.The evaluated models include Gemma4-E2B-it, Qwen3-4B variants, Gemma3-12B-it, Qwen3 models up to 32B, and Qwen2.5-72B-Instruct.
  • Teacher: 11,546 task instances were collected from 3,553 unique training tasks using 3–4 teacher attempts per task.Oversampling was intended to produce diverse solution paths and reduce overfitting to a single trajectory per task; the teacher achieved a 99.0% cumulative success rate.
  • LoRA Training Details: LoRA training searches ranks r ∈{8, 16, 32} and learning rates lr ∈{10−6, 5 × 10−6, 10−5, 5 × 10−5}, fixing α/r = 2.Models use the full training split with the Full History input strategy, and the best checkpoint is selected by MemP (W/ CD) validation success rate.
  • Metrics: Success rate (%) measures tasks solved within maximum steps, while #Steps and Seconds measure mean interaction rounds and per-task runtime.These metrics respectively capture task completion, interaction length, and time used per task.

5 Results

DuoMem substantially improves compact models’ procedural task success by combining context-space and parameter-space distillation, while also reducing inference time and maintaining low deployment overhead. Ablations show complementary contributions from teacher-generated memories, LoRA training, training-history design, and memory quantity.

  • 5.1 DuoMem performance: 55.7%: Gemma4-E2B-it improves from 2.1% without memory to 55.7%, while Gemma3-12B-it rises from 14.3% to 66.4%.These results demonstrate substantial gains across compact 2B–12B students.
  • 5.3–5.6 Ablations: 39.3 points: For Gemma4-E2B-it, DuoMem exceeds CD alone, while for Qwen3-4B-Instruct it reaches 77.9%, 5.8 points beyond MemP+LoRA, showing complementary distillation spaces.For Gemma4-E2B-it, the full configuration reaches 55.7%, adding 9.3 points over LoRA+MemP and 39.3 points over CD alone.
  • 5.2 Efficiency analysis: 4.89 seconds: DuoMem-enhanced Qwen3-4B-Instruct completes tasks on average in 4.89 seconds, a 3.4× speedup over the teacher’s best configuration, while achieving a 77.9% success rate.Average steps decrease from 29.3 to 20.6, and average task completion time drops from 18.49s to 4.89s.
  • 5.2 Efficiency analysis: 5.9M parameters: Qwen3-4B-Instruct’s rank r=8 LoRA adapter adds 5.9M parameters, while the entire teacher memory store occupies ∼4 MB.The adapter is <0.15% of the 4B backbone, and the MemP configuration typically adds ∼1,200 tokens per task.
  • 5.3–5.6 Ablations: 36.3% relative: CD improves Gemma4-E2B-it in the Script setting, while LoRA alone raises Qwen3-4B-Instruct from 4.3% to 42.1% and full DuoMem reaches 77.9%.CD benefit can diminish for larger models, and LoRA may provide marginal or negative gains for Qwen3-8B.

6 Discussion

Ablations show that CD helps models with poor self-generated memories, whereas LoRA helps models lacking procedural competence; both provide smaller marginal gains for already-capable models. Thinking models may need less structured memory but incur substantial latency, limiting edge deployment.

  • When Does Each Component Help Most?: CD most benefits small or non-thinking models with poor self-generated memories, while LoRA most benefits models lacking procedural competence entirely.For already-capable Qwen3-14B+ models, both components’ marginal gains diminish because their capabilities suffice for effective memory utilization.
  • Thinking vs. Non-Thinking Modes: 82.9%: Qwen3-4B-Thinking achieves this with Trajectory alone, exceeding its 74.3% MemP score.This suggests reasoning capacity can substitute for some structured memory required by non-thinking models.
  • Thinking vs. Non-Thinking Modes: 59–101 seconds per task: Qwen3-8B in thinking mode versus 13–15 seconds without thinking, a 5–7× slowdown.The passage characterizes thinking-mode deployment as impractical on edge devices.

7 Conclusion

DuoMem enables small language models to perform procedural problem-solving through complementary context-space and parameter-space distillation. On ALFWorld, it raises a 4B model’s task success from 4.3% to 77.9% with minimal overhead, approaching the performance of a 72B teacher.

  • Dual-space distillation: DuoMem combines context-space distillation with parameter-space distillation to transfer procedural problem-solving ability to small language models.Context-space distillation enriches inputs with teacher-generated memories, while parameter-space distillation uses LoRA fine-tuning on teacher trajectories.
  • ALFWorld results: 77.9% task success: DuoMem improves a 4B-parameter model from a 4.3% baseline on ALFWorld.The improvement closes most of the gap to a 72B teacher model while adding minimal overhead.

Limitations

DuoMem is evaluated only on ALFWorld, a widely adopted benchmark covering six procedural task categories. Although its architecture- and task-agnostic mechanisms are expected to generalize, broader evaluation in domains such as web navigation and code generation remains to be conducted.

  • Scope of evaluation: DuoMem is evaluated on ALFWorld, a widely adopted embodied-agent benchmark for procedural memory.ALFWorld includes six categories spanning navigation, manipulation, and multi-step object interaction.
  • Scope of evaluation: The framework’s context-space memory transfer and LoRA fine-tuning mechanisms are architecture-agnostic and task-agnostic.These properties motivate the expectation that DuoMem can generalize beyond ALFWorld.
  • Future evaluation: Broader evaluation in procedural domains such as web navigation and code generation is an identified extension.The supplied passage frames these domains as settings for extending the analysis, rather than reporting results there.

Ethical Considerations … D Context Distillation Ablation

The section discusses ethical risks in DuoMem and reports supporting analyses of teacher trajectories, inference results, LoRA hyperparameters, and context distillation across models. These analyses cover teacher retry-based collection, complete test-set metrics, validation-set hyperparameter search, and 11 models ranging from 2B to 72B parameters.

  • Ethical Considerations: Teacher-generated procedural memories may propagate biases or unsafe behaviours from the teacher into student models through both distillation spaces.DuoMem stores interaction-derived memories that can amplify helpful or harmful patterns.
  • Ethical Considerations: Compact autonomous edge models may face limited human oversight because reduced connectivity can constrain human-in-the-loop supervision.The paper encourages safety constraints and alignment mechanisms in the distillation pipeline.
  • A Teacher Trajectory Statistics: 5 retry attempts are allowed when the Qwen2.5-72B-Instruct teacher fails to solve a training task.The teacher is prompted to solve each training task during trajectory collection.
  • A Teacher Trajectory Statistics: Near-perfect coverage is achieved with up to 5 attempts, and most tasks are solved on the first try.These are the reported teacher trajectory collection statistics.
  • B Full Inference Results: Table 7 reports test-set success rate and average number of steps across all models and memory configurations.The passage describes the table as providing complete inference results.
  • C LoRA Hyperparameter Search: LoRA hyperparameters are searched for Qwen3-4B-Instruct using the Full History strategy with bug fixing on the validation set.The search results are presented in Table 8.
  • D Context Distillation Ablation: 11 models from 2B to 72B parameters are evaluated to study context distillation in MemP and Script-Only settings.Table 9 examines how context distillation affects performance across different model sizes.

E Thinking vs. Non-Thinking Mode Analysis

Thinking mode generally improves Qwen3-8B success rates but causes a severe 5–7× latency increase, making it impractical for on-device deployment. DuoMem instead transfers procedural knowledge to a non-thinking student while preserving real-time inference speeds.

  • Accuracy–Latency Trade-off: 77.1% vs. 62.1% on Script+CD shows that thinking mode can substantially improve Qwen3-8B success without LoRA fine-tuning.Table 10 compares thinking and non-thinking modes across all memory configurations.
  • Accuracy–Latency Trade-off: 5–7× slowdown raises average task completion time from 13–15 seconds to 58–101 seconds in thinking mode.Extended internal reasoning chains substantially increase output tokens per action.
  • On-Device Implications: Over a minute versus ∼13 seconds makes thinking-mode Qwen3-8B impractical for on-device deployment.The latency penalty directly limits real-time use.
  • On-Device Implications: DuoMem recovers much of the accuracy gap by distilling procedural knowledge into a non-thinking student through LoRA and teacher-generated memories.This alternative maintains real-time inference speeds.

F Qualitative Example: No Memory vs. MemP vs. DuoMem

A qualitative ALFWorld comparison shows that No Memory fails to learn action prerequisites, MemP improves navigation but retains object-identity errors, and DuoMem completes the task in 14 steps through complementary context- and parameter-space distillation.

  • Setup: The comparison uses the same Qwen3-4B-Instruct student on a representative ALFWorld task, showing each DuoMem component addresses a distinct failure mode.The task is to clean a spatula and put it in a drawer.
  • No Memory: No Memory repeatedly attempts remote object interactions without navigating first, exhausts invalid actions, and fails to locate the spatula.The 4B model receives repeated “Nothing happens” responses and later substitutes a butterknife for the spatula.
  • MemP: MemP learns that navigation must precede interaction and searches systematically, but mistakes a butterknife for the spatula and gives up after 30 steps.Procedural memory improves the strategy, yet the model still fails to open drawers and reason correctly about object identity.
  • DuoMem: 14 steps: DuoMem navigates directly to the spatula, cleans it, opens drawer 1, and places it inside, completing the representative task.The configuration combines teacher-generated procedural memories with LoRA fine-tuning, enabling a near-optimal trajectory.

G Extended Related Work

DuoMem is situated among research on memory mechanisms, procedural workflow induction, on-device LLM deployment, context management, and parameter-efficient adapter adaptation. These lines of work provide complementary perspectives on how agents store, deploy, and adapt knowledge.

  • Memory Mechanisms for LLM Agents: Memory-agent research spans episodic, semantic, and procedural memory, surveyed mechanisms from context buffers to structured systems, and verbal self-reflection.Sumers et al. formalize cognitive memory types, Zhang et al. survey memory mechanisms, and Reflexion adds verbal self-reflection.
  • Procedural Memory and Workflow Induction: Workflow-oriented agents build reusable skills, extract compositional workflows, or share modular memories, complementing DuoMem’s procedural scripts with alternative abstractions.Voyager builds a skill library, AWM induces reusable sub-routines, and LEGOMem composes selectively shared private memory modules.
  • On-Device and Edge LLM and Model Deployment: Edge-LLM research addresses model compression, runtime optimisation, hardware-aware evaluation, and modular task execution under memory constraints.The cited work surveys edge LLMs, benchmarks small language models, and proposes compositional multi-tasking for on-device deployment.
  • Context Management and Continual Learning: Context-management studies show that filled context windows can cause failures and that naive truncation may paradoxically improve performance, motivating DuoMem’s context-space design.This finding concerns retrieved memories, reference trajectories, and multi-turn histories competing for context capacity.
  • Adapter Merging and LoRA Extensions: Recent adapter work studies efficient LoRA composition and continual on-device merging, enabling sequential knowledge accumulation without catastrophic forgetting.K-Merge is specifically relevant when DuoMem students adapt to evolving task distributions.
Loading 2606.29961v1…