Source-linked AI summary
LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Models
Qingyu Ren, Qianyu He, Jingwen Chang, Geng Zhang, Jiajie Zhu, Xingzhou Chen, Zhuofei Shi, Jiaqing Liang, Yanghua Xiao, Han Xia, Zeye Sun, Fei Yu
TL;DR
Instruction-following training must handle multiple constraints linked by logical structures, while existing data construction and reward aggregation often overlook those dependencies. LSRIF builds parallel, sequential, conditional, and nested constraint structures and aligns reward aggregation with their semantics. Experiments report improved instruction following across in-domain and out-of-domain benchmarks, alongside gains in logic reasoning and attention to constraint-related tokens and logical connectors.
Problem
Existing instruction-following methods often combine constraints without modeling their logical relationships and average rewards despite sequential dependencies and inactive conditional branches.
Method
LSRIF constructs nested logic-structured constraint trees and applies structure-aware reward aggregation aligned with parallel, sequential, and conditional execution semantics.
Results
LSRIF significantly improves instruction-following and logic-reasoning ability, with gains across in-domain and out-of-domain benchmarks.
Takeaways & Limitations
Logic-structured training is associated with greater attention to constraint-related tokens and logical connectors, with evidence of transfer to logic reasoning tasks.
Takeaways & Limitations
The study does not evaluate models larger than 70B and uses primarily English training data, limiting validation at larger scale and the strength of its cross-lingual generalization claim.
Abstract
from arXiv · showhide
Instruction following is critical for large language models, yet real-world instructions often involve multiple constraints with logical structures, such as parallel composition, sequential dependencies, and conditional branching. Existing methods typically construct data by simply combining constraints and aggregate rewards by averaging individual constraint scores during training, overlooking logical dependencies and introducing noisy signals. We propose LsrIF, a training framework for logic-structured instruction following. LsrIF constructs data by organizing atomic constraints into parallel, sequential, conditional, and nested structures, and applies structure-aware reward aggregation aligned with their execution semantics: averaging rewards for parallel constraints, decaying later rewards after early failures in sequential structures, and rewarding only active branches in conditional structures. Experiments show that LsrIF improves instruction following in both in-domain and out-of-domain settings while also benefiting logic reasoning. Further analysis indicates that logic-structured training increases attention to constraint-related tokens and logical connectors, suggesting improved modeling of instruction logic. We will release our data and code for future research.
1 Introduction
Real-world instructions combine constraints through parallel, sequential, and conditional logic, but existing training data and rewards often ignore these relationships. LSRIF addresses this by constructing nested logic-structured instructions and aggregating rewards according to their execution semantics.
- Motivation: Complex instructions require satisfying individual constraints while also respecting their logical relationships.Parallel structures require all constraints simultaneously; sequential structures impose dependencies; conditional structures introduce branches.
- Limitations of Existing Methods: Most existing methods construct training instructions by adding constraints directly, overlooking sequential dependencies and conditional branches.Some logic-structured datasets are designed primarily for evaluation rather than training.
- Limitations of Existing Methods: Existing reward methods often average individual constraint rewards, even though early sequential failures can make later constraints irrelevant and inactive conditional branches should not contribute.This produces rewards that do not reflect the instruction’s logical structure.
- LSRIF: LSRIF organizes compatible atomic constraints into recursively composable parallel, sequential, conditional, and nested constraint trees.The resulting trees are rendered into complex natural-language instructions.
- LSRIF: LSRIF applies structure-aware reward aggregation: parallel rewards are averaged, sequential failures propagate penalties, and conditional rewards include only active branches.Aggregation follows the execution semantics of the constraint tree.
2 Related Work
Prior instruction-following work constructs multi-constraint data and uses several training paradigms, but commonly overlooks logical relationships among constraints. LSRIF instead targets logic-structured training data and rewards.
- Datasets: Existing datasets combine multiple constraints to improve instruction following but often ignore logic structures that better simulate real-world instructions.Some datasets model logical structures, but they are primarily intended for evaluation rather than training.
- Training Paradigms: Instruction-following training has progressed from supervised fine-tuning to preference optimization and reinforcement learning with verifiable rewards.These paradigms differ in how they optimize instruction-following behavior.
- Reward Modeling: Existing RLVR methods commonly aggregate constraint-level rewards by simple averaging, which fails to represent sequential or conditional structures.Averaging is appropriate only when constraints behave as parallel components.
3 Method
LsrIF constructs complex instructions as logic-structured constraint trees and evaluates responses with rewards aggregated according to parallel, sequential, conditional, and nested execution semantics.
- Logic-Structured Data Construction: LsrIF transforms seed examples into complex instructions by selecting compatible atomic constraints and composing them into logic-structured constraint trees.The trees can contain parallel, sequential, conditional, and nested nodes, then are rendered as natural-language instructions.
- Constraint-Level Evaluation: Each response is evaluated on every atomic constraint using deterministic verifiers for hard requirements and reward models for soft requirements.Leaf rewards are binary and are subsequently aggregated through the constraint tree.
- Structure-Aware Reward Aggregation: Parallel nodes average the rewards of their equally required child constraints.This aggregation is applied recursively within the constraint tree.
- Structure-Aware Reward Aggregation: Sequential nodes penalize later constraints when earlier constraints fail, with smaller λ producing stronger penalties.The penalty factor controls how strongly earlier failures propagate to later rewards.
- Structure-Aware Reward Aggregation: Conditional nodes evaluate the branch condition on the input and compute reward from the activated branch only.Nested nodes recursively apply the aggregation rule of each logical substructure before passing rewards to the parent.
- RL Optimization: The resulting root reward is normalized within response groups and used as the training signal for GRPO policy optimization.Higher structured rewards receive larger positive advantages, while a reference-policy regularizer supports training stability.
4 Experiment
Across model scales, LSRIF improves in-domain and out-of-domain instruction following, logical reasoning, and several general capabilities. Ablations and interpretability analyses support the importance of logic-structured data, structure-aware rewards, and attention changes focused on logical and constraint tokens.
- Main Results: LSRIF improves instruction-following performance across model families and scales on both in-domain and out-of-domain benchmarks.Compared with Base, gains reach +25.2 on IFEval, +11.0 on CFBench, +7.0 on FollowBench, +6.5 on ComplexBench, +6.3 on Collie, and +8.7 on AgentIF.
- Main Results: Qwen2.5-1.5B-Instruct improves from 43.6 to 68.8 on IFEval and from 42.8 to 51.5 on AgentIF after LSRIF training.The gains are particularly pronounced for smaller or weaker models.
- Main Results: LSRIF scores higher than SFT across all evaluated model–benchmark combinations and enables Qwen3-8B to exceed listed baselines on several benchmarks.Qwen3-8B reaches 90.2 on IFEval, 79.2 on ComplexBench, and 65.0 on AgentIF.
- Logical Reasoning: LSRIF improves logical reasoning across all reported Enigmata subcategories, with especially large Arithmetic gains for Distill-Qwen-7B and Distill-Qwen-14B.Distill-Qwen-7B improves from 3.7 to 14.3 (+10.6), while Distill-Qwen-14B improves from 21.0 to 39.0 (+18.0).
- Ablation Studies: Ablations show that removing logic-structured data, structure-aware reward aggregation, penalty propagation, or branch selection reduces performance.Removing logic-structured data causes drops of 2.4 on IFEval, 2.0 on CFBench, and 6.6 on Enigmata-Arithmetic; removing structure-aware reward causes drops of 3.9, 5.0, and 3.6 on the three benchmarks.
- Interpretability Analysis: After training, parameter updates vary by layer and attention module, while token saliency increases concentrate on logical connectors and constraint-related tokens.Lower layers show larger changes in several attention and MLP projections; upper-layer key projections change more, and reasoning examples emphasize operators such as “or” and “and”.
5 Conclusion
LSRIF models logical relationships among constraints during data construction and reward design, covering parallel, sequential, conditional, and nested structures. Experiments report improved instruction following and logic reasoning, associated with greater attention to constraint-related tokens and logical connectors.
- LSRIF explicitly models parallel, sequential, conditional, and nested relationships among constraints in data construction and reward design.
- Structure-aware reward aggregation follows the execution semantics of logic-structured constraints.
- LSRIF significantly improves instruction-following ability and logic reasoning ability.
- Interpretability analysis associates these gains with more attention to constraint-related tokens and logical connectors.
Limitations
The paper identifies limited evaluation scale and primarily English training data as its main limitations. It notes that larger-model validation and multilingual logic-structured data would strengthen robustness and cross-lingual generalization claims.
- The method is not evaluated on larger models such as 70B+ because of computational constraints.
- Validation on larger models would further strengthen the credibility and robustness of the approach.
- The training data is primarily English, limiting the direct scope of the cross-lingual generalization claim.
- Multilingual logic-structured instruction data would further strengthen evaluation of cross-lingual generalization.
B.3 Reward Model Training
The reward model is fine-tuned to predict whether responses satisfy atomic constraints, and its predictions show reasonable agreement with human judgments.
- The reward model is fine-tuned for binary classification of whether a response satisfies a given constraint.
- The evaluation uses 600 test samples with three atomic soft constraints each and 1,800 human annotations.
- 0.87 Precision, 0.84 Recall, and 0.86 F1-score indicate reasonable agreement between reward-model predictions and human judgment.
C Benchmarks
The evaluation spans instruction following, logical reasoning, general reasoning, knowledge, and conversational ability benchmarks. These benchmarks cover automatically verifiable constraints, complex compositions, agentic scenarios, multi-turn dialogue, and graduate-level reasoning.
- IFEval evaluates automatically verifiable instruction compliance across 25 rule-based constraint types and around 500 prompts.
- CFBench covers 1,000 samples across over 200 real-life scenarios and more than 50 NLP tasks.
- FollowBench increases difficulty by adding constraints across Content, Situation, Style, Format, and Example types.
- ComplexBench assesses complex instructions using a taxonomy of 4 constraint types, 19 constraint dimensions, and 4 composition types.
- AgentIF evaluates instruction following in agentic scenarios drawn from 50 real-world applications, averaging 11.9 constraints per instruction.
- GPQA-Diamond contains 198 graduate-level questions in biology, chemistry, and physics, while AIME2024 and AIME2025 evaluate mathematical reasoning.
- Enigmata evaluates logical reasoning through 36 tasks across seven categories with automatic generators and rule-based verifiers.
- MT-Bench evaluates multi-turn conversational ability with 80 open-ended questions across diverse topics.
D Dataset
The dataset distinguishes soft constraints, which require subjective judgment, from hard constraints that can be checked by explicit rules. It also provides a template for adding atomic constraints to seed instructions.
- Soft constraints target subjective properties such as semantic focus, tone, emotion, style, audience, authorship, and syntax.
- The dataset uses a prompt template to add atomic constraints to relatively simple seed questions and make their instructions more complex.
- Hard constraints specify rule-checkable requirements involving keywords, lengths, formats, placeholders, markers, or punctuation.
E Sensitivity Analysis on Penalty Factor
The sensitivity analysis selects λ = 0.5 as the best-performing penalty setting, while additional analyses examine nested structures, training dynamics, and parameter adaptations across models.
- Sensitivity Analysis on Penalty Factor: λ = 0.5 provides the appropriate penalty and achieves the best performance with Qwen2.5-7B-Instruct.
- Nested Logical Structures: 14.4% improvement was achieved by Distill-Qwen-14B on Selection_and_Chain_3 after training.Selection_and_Chain_3 represents nested structures with depth 3.
- Training Dynamics: Rewards increase early and then stabilize with moderate oscillations, while response lengths remain variable without consistent growth.Qwen2.5-7B-Instruct improves rapidly, whereas Distill-Qwen models and Qwen3-8B show steadier or more fluctuating gains.
- Parameter Changes: Parameter updates vary by module and layer, with stronger heterogeneity in attention-related projections than in MLP modules.Query and key projections often change more in middle-to-upper layers, while value and output changes are more model-specific.
I Full Token-Level Information Flow Analysis
Token-level analysis compares information flow before and after training, showing more concentrated importance on logical cues and constraint-related tokens after LSRIF training. The analysis uses qualitative visualizations across Distill-Qwen-7B and Qwen3-8B.
- Before Training: Before training, both models show relatively diffuse token-importance patterns across content and constraint-related words.
- After Training: After training, Distill-Qwen-7B assigns stronger importance to ordering and conditional cues such as “First,” “then,” “if,” and “else.”
- After Training: Qwen3-8B similarly highlights logical and constraint-related tokens, including ordering cues, formatting terms, and style requirements.
- Interpretation: The qualitative examples suggest that logic-structured training helps models identify and track logical relations, ordering requirements, and formatting constraints.This trend is observed in both Distill-Qwen-7B and Qwen3-8B.