PaperlayerView exact revision on arXiv

Source-linked AI summary

Logic Distillation: Learning from Code Function by Function for Decision-making Tasks

Dong Chen, Shilin Zhang, Fei Gao, Yueting Zhuang, Siliang Tang, Qidong Liu, Mingliang Xu

arXiv:2407.19405v2cs.AI

TL;DR

Smaller LLMs are easier to deploy than larger models but may struggle with complex decision rules and output-based distillation. Logic Distillation represents task logic as functions that smaller models select during decisions; the paper reports results comparable to or better than larger models in its 21-points setting.

  • Problem

    Smaller LLMs can struggle to follow complex decision rules, and output-mimicking students may lose decision-making capability in unfamiliar scenarios.

    Source: §1 · p. 2

  • Method

    Logic Distillation uses larger LLMs to convert instructions into functions, builds a function base, and trains smaller models to select and invoke functions for decisions.

    Source: §1 · p. 2

  • Results

    In 21 Points, S-LLMs with LD achieve results comparable to or even superior to L-LLMs.

    Source: §4.4 · p. 7

  • Takeaways & Limitations

    The paper concludes that representing decision logic as functions can let smaller models maintain a global perspective in decision-making.

    Source: §4.4 · p. 7

    The described generation analysis assumes M token types and K function types, limiting that analysis to this setup.

    Source: §3.5 · p. 4

Abstract

from arXiv · show

Large language models (LLMs) have garnered increasing attention owing to their powerful logical reasoning capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful logical reasoning capabilities. Consequently, S-LLMs are helpless when it comes to planning and decision-making tasks that require logical reasoning capabilities. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, based on the function base, LD fine-tunes S-LLMs to learn the logic employed by L-LLMs in planning and decision-making. During testing, LD utilizes a retriever to identify the top-$K$ relevant functions based on instructions and current states, which will be selected and invoked by S-LLMs. Ultimately, S-LLMs yield planning and decision-making outcomes, function by function. Relevant experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in planning and decision-making tasks, comparable to, or even surpassing, those of L-LLMs.

1 Introduction

Larger LLMs offer stronger logical reasoning but are difficult to deploy, while smaller models often struggle to follow complex rules and learn decision logic from output imitation. Logic Distillation represents decision logic as functions that smaller models select and invoke at each stage.

1 Introduction

  • L-LLMs are difficult to deploy on most devices, whereas S-LLMs can run on consumer-grade GPUs but generally perform worse, particularly in logical reasoning.Source: §1 · p. 1
  • KD students can mimic teacher outputs yet violate complex rules and lose decision-making capability in unfamiliar scenarios.Source: §1 · p. 2
  • LD decomposes task rules into functions, provides usage comments and examples, and trains S-LLMs to select and invoke relevant functions at each decision stage.Source: §1 · p. 2
  • Unlike KD's output imitation, LD focuses on representing task-execution logic as functions that students use to make decisions.Source: §1 · p. 2, Figure 2 · p. 2
  • The paper identifies decision-making challenges for KD and proposes LD, with experiments across different scenarios to assess its effectiveness.Source: §1 · p. 2

2 Related Work

Prior distillation methods transfer teacher outputs or reasoning-oriented supervision, but the paper characterizes them as still relying on imitation. LD instead represents decision logic as functions for students to use.

2 Related Work

  • Large models' size can make them impractical to run on most devices and costly to invoke through interfaces.Source: §2 · p. 2
  • Conventional KD trains smaller students to mimic larger teachers, while reasoning-focused approaches add rationales or solution-strategy supervision.Source: §2 · p. 2
  • Logic Distillation (LD) uses L-LLMs to decompose human-provided rules into basic functions for a function base, while S-LLMs choose functions for task stages and make decisions function by function.Source: Figure 3 · p. 3
  • Orca and Orca 2 add reasoning-process prompts or strategy selection, but the paper describes these approaches as still having students mimic teacher outputs.Source: §2 · p. 3

3 Methodology

LD converts instructions into staged code functions and a function base, retrieves relevant candidates from instructions and current states, then has S-LLMs select and apply functions for decisions.

3 Methodology

  • Interactive decision-making is organized into steps and stages; LD decomposes instructions into functions, retrieves relevant candidates, and has S-LLMs select functions to make decisions.Each interaction counts as one step, and several stages can collectively complete one decision-making process.Source: §3 · p. 3

3.1 L-LLMs

  • L-LLMs generate functions and user manuals token by token, and the collected functions and manuals form the function base Df.The manuals include rule explanations, code comments, and corresponding invocation stages.Source: §3.1 · p. 3

3.2 Retriever

  • For large function bases, LD uses DPR retrieval; for small bases, it fine-tunes S-LLMs to select and use functions directly from Df.The large-base retriever returns the top-K functions relevant to instructions and current states.Source: §3.2 · p. 3

3.3 S-LLMs

  • At each stage, S-LLMs select and execute a function for an intermediate result, repeating across J stages before continuing or halting when task requirements are met.If the output does not meet the requirements, it becomes input for the next step.Source: §3.3 · pp. 3–4

3.4 Emergency handling of S-LLMs in LD

  • For emergencies, LD turns xE into candidate functions; the paper says this gives S-LLMs stronger general capabilities, including function generation.The framework contrasts remembering function usage with memorizing large amounts of L-LLM outputs.Source: §3.4 · p. 4

3.5 Why Selection Is Better

  • With M token types and K function candidates where K << M, the analysis argues selection has lower maximum entropy and a higher lower bound, supporting more stable outputs than generation.The comparison concerns the maximum entropy of selection versus generation.Source: §3.5 · p. 4

4 Experiments

The experiments evaluate Logic Distillation (LD) in pursuit, emergency-response, and 21-points tasks, and examine why it works. Across these settings, the reported results favor LD over knowledge distillation (KD) and, in some comparisons, larger LLMs.

4. Experiments

  • The experiments aim to test LD’s decision-making effectiveness, explain its effectiveness, and assess emergency response across several L-LLM and S-LLM model pairs.The listed models are GLM-4, LLaMA3-70B, and Qwen2.5-72B as L-LLMs, and GLM4-9B, LLaMA3-7B, and Qwen2.5-7B as S-LLMs.Source: §4 · pp. 4–5
  • LD builds a function base from L-LLM-generated functions and a user manual, then fine-tunes S-LLMs to select and use functions in decision-making.The manual explains functions, comments, and invocation stages.Source: Figure 4 · p. 5, §4.1 · p. 6
  • The pursuit-game results compare L-LLM, S-LLM, S-LLM with KD, and S-LLM with LD on 200 starting-position sets, with games capped at 100 moves and failure after more than seven illegal choices.The game concludes when the Manhattan distance between all three blue dots and the orange dot is less than 2 units.Source: §4.1 · p. 5, Table 1 · p. 6

4.1 Better Performance in Pursuit Game

  • LD reaches 100% success in the pursuit game, with fewer average steps than L-LLMs.The authors attribute the results to LD improving S-LLMs’ decision-making capabilities.Source: §4.1 · p. 6, Table 1 · p. 6

4.2 Why LD is Better

  • LD’s pursuit trajectories approach the target more directly, while KD can produce repetitive circling and L-LLM can backtrack.The authors attribute LD’s advantage to functions designed from a global perspective, whereas L-LLM decisions can be influenced by the current state.Source: Figure 5 · p. 5, §4.2 · p. 6

4.3 Pursuit Game with Emergencies

  • In the emergency scenario, LD’s success rate exceeds L-LLM’s by 10%, while KD reaches 52% and L-LLM reaches 90%.LD adds a coordinate-filtering function for the restricted area; the authors also report that it uses only a few functional examples, helping S-LLMs retain more general capabilities.Source: §4.3 · p. 6, Table 2 · p. 7

4.4 Better Performance in 21 Ponits

  • In 21 points, LD-equipped S-LLMs achieve results comparable to or better than L-LLMs, while KD’s improvement is described as not significant.The authors relate LD’s performance to teaching decision logic through functions and retaining a global perspective.Source: §4.4 · p. 7, Table 3 · p. 7, Figure 7 · p. 7

5 Conclusion

Logic Distillation (LD) converts L-LLM instructions into functions and fine-tunes S-LLMs to use them for decisions. The paper reports that a small amount of example fine-tuning can let S-LLMs match or surpass L-LLMs on decision-making tasks.

5 Conclusion

  • LD addresses the gap between deployable S-LLMs and L-LLMs by targeting decision-making, which the authors say KD’s output mimicry does not adequately address.The paper describes L-LLMs as difficult to deploy on most devices and S-LLMs as easier to deploy but weaker in performance.Source: §5 · p. 7
  • LD instantiates L-LLM logic as functions, establishes a function base, and fine-tunes S-LLMs to comprehend and use those functions for decisions.Source: §5 · p. 7
  • The authors report that a small amount of example fine-tuning enables S-LLMs to match or even surpass L-LLMs in decision-making capabilities.Source: §5 · p. 7

Related papers

Loading 2407.19405v2…