PaperlayerView exact revision on arXiv

Source-linked AI summary

Small Reasoning Models are Instruction Followers in Function Calling

Yalda Taheri, Mohammad Hassan Heydari, Erfan Naaman, Afsaneh Fatemi

arXiv:2608.22472v1cs.AIcs.CL

TL;DR

Function calling is challenging for compact models because native tool contexts impose rigid syntactic constraints. IFFC decouples function-calling logic into a dedicated instruction-following model, and the resulting approach improves accuracy, including under aggressive quantization.

  • Problem

    Compact models face function-calling challenges because native tool contexts require rigid tags and JSON schemas that are brittle and formatting-sensitive.

  • Method

    IFFC separates function calling from the primary question-answering model and delegates intent evaluation, routing, and tool execution to a dedicated Small Reasoning Model using instruction following and ReAct.

  • Results

    IFFC enables compact models such as Qwen-3 4B to outperform massive proprietary baselines, while instruction-following contexts achieve higher tool-selection accuracy than native tool-calling contexts.

  • Takeaways & Limitations

    Aggressive quantization preserves robust reasoning-model performance, supporting efficient, privacy-preserving deployment of agentic capabilities on resource-constrained edge devices.

  • Takeaways & Limitations

    The decoupled two-stage architecture requires sequential inference, introducing latency and scalability trade-offs for real-time applications with strict latency budgets.

Abstract

from arXiv · show

Function calling represents the core capability of agentic large language models (LLMs). Existing research has focused on enhancing LLMs function-calling accuracy through fine-tuning, reinforcement learning (RL), and multi-agent frameworks, particularly for native function-calling LLMs. This work demonstrates that LLMs achieve superior accuracy in function calling in instruction-following contexts (i.e., standard user-assistant interactions) rather than a tool calling context. We introduce Instruction-Followed Function Calling (IFFC), a novel framework that decouples function-calling logic from the primary LLM and delegates it to a dedicated smaller model operating within the instruction-following paradigm. Our method consistently outperforms both native function calling (NFC) and prompt-based function calling (PFC) baselines, with particularly strong gains on reasoning-oriented LLMs. Furthermore, we demonstrate that IFFC maintains robust performance under aggressive quantization, enabling efficient on-device deployment without significant accuracy degradation. This work establishes a new paradigm for reliable, resource-efficient function calling in edge-computing scenarios.

1 Introduction

Agentic AI relies on function calling to select tools, generate arguments, and execute actions, but smaller models struggle with rigid syntactic constraints. IFFC addresses this by separating function calling from the primary QA model and using instruction-following contexts.

  • Function calling enables agentic models to select tools, generate arguments, and execute actions in external environments.
  • Small Language Models range from 0.5 to 15 billion parameters and offer lower computational demands, latency, and privacy risks than massive models.
  • IFFC decouples function calling from the primary question-answering model and delegates it to a dedicated instruction-following model.
  • Models achieve superior function-calling accuracy in instruction-following contexts than in native tool-calling contexts.
  • IFFC uses prompt-based function calling and enables smaller reasoning models to execute complex tool usage separately from the main QA LLM.
  • IFFC maintains robust performance under aggressive quantization for deployment on resource-constrained edge devices.

2 Related Works

Prior work seeks to improve small-model tool use through training-based optimization and training-free prompting strategies. These approaches target the performance gap between compact models and larger systems.

  • Small language models under 7 billion parameters are positioned as stable, cost-effective, and private alternatives for structured workflows.
  • Training-based methods use reinforcement learning, GRPO, domain-specific fine-tuning, and planning distillation to reinforce structured tool-use behavior.
  • Training-free methods use natural-language tool interfaces, prompt engineering, and reasoning-oriented strategies to improve tool use.

3 Methodology

IFFC separates tool-selection logic from the primary generation model and reframes function calling as instruction following. A dedicated SRM evaluates intent, executes tools when needed, and returns enriched context to the main model.

  • 3 Methodology: IFFC introduces a two-stage paradigm that decouples tool selection from primary generation and reframes function calling as instruction following.
  • 3 Methodology: A monolithic LLM can experience context pollution when complex API schemas interfere with conversational performance or reasoning depth.
  • 3.1 Decoupling Function Calling from the Main LLM: A dedicated SRM evaluates incoming-query intent to determine whether external context or tool execution is required.
  • 3.1 Decoupling Function Calling from the Main LLM: The SRM handles tool-definition parsing while the main LLM remains focused on user interaction and final response generation.
  • 3.1 Decoupling Function Calling from the Main LLM: IFFC limits SRM context growth because the main LLM’s several-hundred-token responses are not appended to the SRM’s conversational history.
  • 3.1 Decoupling Function Calling from the Main LLM: IFFC uses PFC for intent evaluation, executes required functions, and appends resulting context directly to the query.
  • 3.2 Instruction-Followed Function Calling (IFFC): Rigid native tool constraints can cause formatting errors and hallucinations in smaller models, whereas instruction prompting supports function calling.
  • 3.2 Instruction-Followed Function Calling (IFFC): IFFC treats tool definitions as high-priority system instructions and function calls as standard assistant responses instead of specialized tokens.

4 Experiments

The experiments compare IFFC with NFC and PFC across open-weight models, proprietary baselines, reasoning modes, and quantization levels. They evaluate competitiveness, reasoning effects, and edge-deployment robustness.

  • 4 Experiments: The evaluation compares IFFC against NFC and PFC across performance, reasoning-mode effects, and quantization robustness.
  • 4 Experiments: The benchmark includes Gemma-3, Phi-4 Mini, Qwen-3, Granite 4 Micro, and Granite 4 Tiny-h open-weight models.
  • 4 Experiments: IFFC-operated SRMs are compared with GPT 5.2, Gemini 2.5 Pro, and Claude 4.5 Sonnet in native function-calling modes.
  • 4 Experiments: Qwen-3 hybrid models isolate the contribution of explicit reasoning traces to argument parsing and schema adherence.
  • 4 Experiments: The study compares FP16 and Q4KM models to test whether IFFC performance is maintained after compression.
  • 4 Experiments: Gemma-3 models are used for a direct IFFC-versus-PFC comparison because both methods rely on natural-language prompts.

5 Results

IFFC consistently outperforms NFC and PFC baselines, with especially strong gains for smaller reasoning models and complex tool-use scenarios. Reasoning-enabled models also show substantial gains and resilience under aggressive quantization.

  • IFFC consistently outperforms the NFC baseline across Qwen-3 models from 0.6B to 8B and Phi-4 Mini.
  • 44.3%: Phi-4 Mini accuracy in Live evaluation with IFFC, compared with approximately 30% in NFC mode.
  • 79.1%: Gemma-3 12B accuracy on Parallel Multiple queries with IFFC, while Gemma-3 4B using PFC drops to near 0%.The passage attributes the PFC degradation to context drift and hallucination, while linking IFFC robustness to decoupled routing.
  • 96.0% and 97.5%: Qwen-3 4B IFFC accuracy on Non-Live Simple and Multiple tasks, exceeding GPT-5.2 and matching or exceeding Claude 4.5 Sonnet.
  • 57.7%: Qwen-3 0.6B Live accuracy with Think mode, up from 22.9% without reasoning.For Qwen-3 4B, Think mode increases accuracy from 74.6% to 86.7%.
  • 93.5%: Qwen-3 4B Think accuracy at Q4KM, versus 94.1% at FP16, while the 8B Think model reaches 94.4%.The paper interprets reasoning traces as error correction that compensates for weight-precision loss.

6 Conclusion

The conclusion presents IFFC as a framework that prioritizes natural-language instruction adherence and decouples routing from the main QA model. It reports strong performance from compact reasoning models and robustness under aggressive quantization for edge deployment.

  • IFFC redefines SRM-based agentic task execution by prioritizing natural-language instruction adherence over rigid native tool definitions.
  • Reasoning-oriented models remain robust under aggressive quantization, supporting efficient, privacy-preserving edge deployment.

7 Limitations

IFFC’s decoupled two-stage architecture improves routing performance but introduces latency and scalability trade-offs. Sequential inference can delay real-time applications even when SRM quantization reduces resource overhead.

  • Sequential inference in the decoupled two-stage architecture introduces inherent latency and scalability trade-offs.
  • Real-time applications with strict latency budgets may still experience delays despite aggressively quantizing the SRM to Q4KM.

A IFFC and PFC: A review on differences

PFC enables function calling for models without native function-calling capabilities by encoding tool use through natural-language prompting. Unlike NFC, PFC emits invocations in the standard assistant response role rather than a specialized tool-use role.

  • PFC supports models lacking native function-calling capabilities, including Gemma-3, by defining tools and invocation behavior through a system prompt.
  • PFC serializes available tools into the system context and instructs the model to invoke them when necessary.
  • PFC generates function invocations in the standard assistant response role, whereas NFC uses a specialized tool_use role.

A.2 Relation to IFFC

IFFC retains prompt-based function calling’s instruction-following basis while moving tool selection from the main conversational model to a dedicated SRM. It further adapts output formatting and prompting to improve reliability under rigid function-call evaluation.

  • Decoupled Architecture: IFFC offloads function calling from the main conversational model to a dedicated Small Reasoning Model, isolating the tool-use context.This decoupled architecture distinguishes IFFC from standard prompt-based function calling, which normally supplies tool definitions to the main model.
  • Customized Formatting: IFFC enforces a specialized output format instead of the generic JSON format commonly required by function-calling benchmarks.The format is designed around the SRM’s reasoning capabilities.
  • Prompt Adaptation: Iterative BFCL testing led IFFC to add targeted system-prompt hints that guide the SRM’s function-calling logic.These hints address weaknesses revealed by rigid evaluation criteria.
  • Prompt Adaptation: The IFFC prompt requires parameter values to follow the exact formats illustrated in function docstrings.For example, a weather API example using “City, Country” guides the model toward “Paris, France” rather than “Paris.”
  • Workflow: The workflow constructs an SRM input from its system prompt, available functions, query, and SRM history, then generates a PFC-based response.The provided workflow also evaluates whether the SRM output triggers a function call and analyzes whether external knowledge is required.
  • Workflow: IFFC preserves context separation by excluding the main LLM’s final response from the SRM’s memory.This design is intended to keep the function caller less context polluted by the QA model’s outputs.

14: // Stage 2: Independent Response Synthesis via Main QA LLM

After the SRM handles function routing, the main QA LLM independently synthesizes the final response from the enriched query and its own conversational state. The SRM maintains a separate memory containing the user query and its generated output, but not the final response.

  • 14: // Stage 2: Independent Response Synthesis via Main QA LLM: The main LLM receives a context-enriched query and its own conversational state to construct the response-synthesis input.The enriched query is combined from the original query and contextual information before main-model prompting.
  • 14: // Stage 2: Independent Response Synthesis via Main QA LLM: The SRM’s memory is updated with the user query and SRM output while excluding the final response.This preserves the SRM’s separate context space from the main QA model’s synthesis.
  • 14: // Stage 2: Independent Response Synthesis via Main QA LLM: The SRM system prompt defines function-calling decisions, available functions, and the required JSON output structure.Queries without required function calls produce an empty list; otherwise, each call includes a function name and parameter dictionary.

C Full Results

The full-results evaluation compares IFFC, including FP16 and Q4KM variants, with PFC and NFC across BFCL categories and model settings. The reported tables also document model-support, resource, timing, and presentation conditions affecting coverage.

  • C Full Results: IFFC results are compared with Prompt Function Calling and Native Function Calling across BFCL evaluations, including FP16 and Q4KM variants.The comparisons cover the reported Simple, Multiple, Parallel, and Parallel Multiple categories in live and non-live settings.
  • C Full Results: Gemma-3 models lack NFC support, so their NFC entries are marked with “-”.This makes NFC comparisons unavailable for those model rows.
  • C Full Results: Granite-4 models were evaluated primarily in Q4KM because of resource constraints.The quantized setting therefore represents most Granite-4 results in the reported evaluation.
  • C Full Results: IFFC FP16 results are omitted for most models in Live Parallel and Live Parallel Multiple because of time constraints during evaluation.Those categories therefore have incomplete FP16 coverage across models.
  • C Full Results: The reported tables separately cover non-live Simple and Multiple, non-live Parallel and Parallel Multiple, live Simple and Multiple, and live Parallel and Parallel Multiple evaluations.These table groupings organize the full-results comparison by evaluation mode and task category.
Loading 2608.22472v1…