PaperlayerView exact revision on arXiv

Source-linked AI summary

PTC-Decoder: Towards Intelligent SLMs on Offline Resource-Constrained Edge Devices

Minghui Yu, Ke Mu, Gang Wu

arXiv:2609.30836v1cs.AI

TL;DR

Offline, resource-constrained satellite agents need reliable SLM instruction-following, but training data are scarce and prompt-based planning does not ensure stable planning. PTC-Decoder combines planning-first execution with constrained tool decoding. Across seven SLMs, it achieves a mean overall gain of +1.21; final-answer accuracy remains an open challenge.

  • Problem

    Offline, resource-constrained satellite agents need reliable SLM instruction-following, while training data are scarce and prompt-based planning cannot ensure stable planning.

    Source: Introduction · pp. 1–2

  • Method

    PTC-Decoder forces a first-round plan-tool call and constrains tool-name outputs while leaving parameter generation to the model, without requiring training.

    Source: Introduction · p. 2, Plan-to-Act · p. 3

  • Results

    PTC-Decoder achieves a +1.21 mean overall score gain across 7 SLMs, with peak F1 of 0.359.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    PTC-Decoder provides a lightweight approach to improving step-level reliability for SLM agents on offline edge devices.

    Source: Conclusion · p. 7

    Final-answer accuracy remains an open challenge because TC-Decoder constrains tool names rather than ensuring correct answers or robust error recovery.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge: their limited reasoning capacity hinders reliable execution of multi-step agent tasks requiring complex tool orchestration. Existing plan-solve paradigms rely on prompt-based enforcement, which our experiments show SLMs almost entirely disregard: weak models fail to invoke the plan. We propose PTC-Decoder (Plan-Tool Constrained Decoder), a training-free, plug-and-play decoder framework that combines (1) a Plan-to-Act paradigm, which elevates planning to an atomic tool and forces its invocation at the first inference step, and (2) TC-Decoder, a deterministic finite automaton that imposes token-level hard constraints on tool names while preserving freedom over parameter generation, thereby retaining SLM reasoning capability. Evaluated on 200 real remote-sensing satellite tasks across 7 SLMs, PTC-Decoder yields a statistically significant mean overall score gain of +1.21 (p<0.01), 95% CI [+1.13, +1.29]), with consistent improvements across models and other datasets. An ablation study that removes TC-Decoder causes substantial performance degradation across all quality metrics without reducing computational cost, confirming TC-Decoder as the primary driver. PTC-Decoder thus offers a lightweight yet effective solution for improving step-level reliability, with final-answer accuracy remaining an open challenge. In essence, we enforce plan adherence by constraining the permissible output vocabulary during inference, without requiring retraining.

Introduction

Offline satellite agents must execute complex tasks with SLMs under tight resource constraints, while existing training- and prompt-based approaches are ill-suited to this setting. PTC-Decoder combines planning-first execution with constrained decoding and improves performance across seven SLMs.

Introduction

  • Less than 4 GB remains for SLM deployment after remote-sensing workloads, while offline satellites cannot access cloud-based LLMs.The paper describes edge devices with unified memory of 16 GB or less and competing resource-intensive applications.Source: Introduction · pp. 1–2
  • Training-based approaches rely on scarce remote-sensing agent data, while prompt-only planning cannot ensure stable planning in SLMs.The prompt-plan comparison reports that weaker models ignore the prompt entirely (P.R.=0).Source: Introduction · p. 1, Table 1 · p. 2
  • The paper frames its target as improving SLM instruction-following on offline, resource-constrained devices where existing research leaves a gap.It identifies offline resource-constrained scenarios as an unresolved setting for SLM instruction-following.Source: Introduction · p. 2
  • PTC-Decoder combines planning-first execution with TC-Decoder, which enforces tool-name outputs at designated steps while leaving parameters to the model.The framework is plug-and-play and requires no training.Source: Introduction · p. 2
  • Across 7 SLMs, PTC-Decoder gains +1.21 mean overall score; the weakest model improves by 350%, and capable models reach F1 0.359.The reported evaluation uses satellite-agent tasks and ground-truth tool sequences.Source: Introduction · p. 2

Related Work

Related work covers on-device SLM deployment, model and inference optimization, and edge-cloud collaboration. The paper positions offline satellite operation as a setting where cloud-dependent approaches cannot be used.

Application of SLMs on Edge Devices

  • Edge SLM research spans hardware acceleration, edge-cloud collaboration, model optimization, and deployment optimization, with thermal throttling a challenge during continuous inference.An empirical study of over 60 SLMs found remaining room to optimize in-context learning and operational efficiency.Source: Application of SLMs on Edge Devices · p. 2

Lightweight Architectures and Model Quantization

  • Lightweight Transformer architectures and quantization address edge deployment constraints; quantization compresses models by 4–10× while retaining 75%–96% accuracy.MobiLoRA instead focuses on LoRA-adapted LLM inference and KV Cache storage constraints on mobile devices.Source: Lightweight Architectures and Model… · p. 2

Edge-Cloud Collaboration

  • Edge-cloud collaboration offloads complex tasks while keeping simple ones local, but offline satellites without network access cannot use cloud-based LLMs.The paper identifies intelligence deficiencies on non-networked edge devices as a largely unexplored challenge.Source: Edge-Cloud Collaboration · p. 2

Methodology

PTC-Decoder targets unreliable multi-step tool use by resource-constrained SLMs through a Plan-to-Act workflow and selective decoding constraints. It forces tool invocation while leaving action-parameter generation to the SLM.

Methodology

  • SLMs with at most 2B parameters face complex remote-sensing tasks that make correct end-to-end agent execution nearly infeasible.The method targets step-level tool-calling adherence in edge-agent tasks.Source: Methodology · p. 3

Plan-to-Act

  • Plan-to-Act makes planning an atomic tool invoked first, then calls the planned tools in order across subsequent agent loops.The plan tool takes a natural-language task and returns a structured sequence of atomic tool names.Source: Plan-to-Act · p. 3

Tool-Constrained Decoder

  • The decoder injects the desired tool at the designated step without changing agent implementations or stable prompts, or requiring additional training.The paper contrasts this plug-and-play approach with SFT or RL as less intrusive.Source: Tool-Constrained Decoder · p. 4
  • Tool-Constrained Decoder hard-constrains tool-invocation formats and names while leaving parameter content to the SLM’s contextual reasoning.It can be enabled on demand and disabled for unconstrained steps.Source: Tool-Constrained Decoder · p. 4
  • A deterministic finite automaton forces preset prefix tokens, then permits any vocabulary token in its absorbing free state until EOS.The automaton transitions from fixed-prefix states to the free state, which loops on every token.Source: Tool-Constrained Decoder · p. 5

Experiments

Across the reported evaluations, PTC-Decoder improves judge scores, benchmark tool metrics, and external-dataset results. The ablation identifies TC-Decoder as the decisive component, with nearly unchanged token overhead.

Experimental Setup

  • Experiments use 7 SLMs with at most 2B effective parameters and 200 expert-annotated real onboard tasks on a Jetson Orin NX.Tasks run alongside remote-sensing algorithms and daemon processes to approximate resource-limited onboard conditions.Source: Experimental Setup · p. 5

LLM-Judge Evaluation

  • The strongest baseline model, Qwen3-1.7B, achieves Ov. 1.98, while weaker models score below 0.75 across all dimensions.Source: LLM-Judge Evaluation · p. 5
  • PTC-Decoder raises mean Ov. by +1.21 across models (p < 0.01); Table 2 reports judge scores across experimental conditions.The table also includes human double-checks for each group.Source: LLM-Judge Evaluation · p. 5, Table 2 · p. 6

Rule-Based Evaluation

Ablation Study

  • In an ablation comparing PTC-Decoder with Plan w/o TC-Decoder, adding TC-Decoder raises Ov. by 0.99 on average across all 7 models (p < 0.01; bootstrap 95% CI [+0.91, +1.06]).Mean benchmark Recall rises from 0.095 to 0.241. The passage describes TC-Decoder as PTC-Decoder’s decisive component and says it does so without additional overhead.Source: Ablation Study · pp. 6–7

Performance on Other Datasets

  • F1 rises from 33% to 68% on Seal-Tools, the largest external-dataset gain, alongside balanced Recall and Precision improvements.Figure 4 shows results across ToolAlpaca, Seal-Tools, and API-Bank, averaged over 7 SLMs.Source: Performance on Other Datasets · p. 6, Figure 4 · p. 7

Ablation Study

  • With TC-Decoder, three models that had zero F1 under free execution reach 0.246, 0.076, and 0.088; Qwen3-1.7B is an exception.Qwen3-1.7B has higher F1 without TC-Decoder (0.355 vs. 0.292), alongside lower Recall (0.252 vs. 0.305).Source: Ablation Study · p. 7

Conclusion

PTC-Decoder combines Plan-to-Act with TC-Decoder to improve SLM agent performance and step-level reliability on offline edge devices. Its gains have limits: answer quality remains weak, and decoding adds token overhead.

Conclusion

  • PTC-Decoder is a training-free, plug-and-play decoder framework that addresses SLM intelligence deficiencies on offline, resource-constrained edge devices.Source: Conclusion · p. 7
  • +1.21 mean Ov. gain across 7 SLMs, with peak F1 of 0.359; removing TC-Decoder reduces average F1 by 0.110 and degrades all quality metrics without an efficiency advantage.The authors identify TC-Decoder as the decisive component.Source: Conclusion · p. 7

Limitations

  • R.A. remains weakest at 2.81/5, Rb. gains average +0.52 versus baseline, and PTC-Decoder incurs 2.07× token overhead.The authors caution that tool-name constraints cannot guarantee correct answers or robust error recovery; Qwen3-1.7B achieves higher F1 without TC-Decoder.Source: Limitations · p. 7
Loading 2609.30836v1…