PaperlayerView exact revision on arXiv

Source-linked AI summary

Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?

Qinyan Zhang, Xinping Lei, Ruijie Miao, Yu Fu, Haojie Fan, Le Chang, Jiafan Hou, Dingling Zhang, Zhongfei Hou, Ziqiang Yang, Changxin Pu, Fei Hu, Jingkai Liu, Mengyun Liu, Yang Liu, Xiang Gao, Jiaheng Liu, Tong Yang, Zaiyuan Wang, Ge Zhang, Wenhao Huang

arXiv:2509.04292v1cs.CL

TL;DR

LLMs can struggle when instructions conflict with conventions learned during fine-tuning, leaving counter-intuitive instruction following underexplored. Inverse IFEval tests this ability with eight categories and 1,012 Chinese-English questions. Experiments report poor performance by fine-tuned Qwen models on non-conventional instructions.

  • Problem

    LLMs can struggle with instructions that deviate from standardized training conventions, while their counter-intuitive instruction-following ability remains underexplored.

    Source: §1 · p. 2

  • Method

    Inverse IFEval tests counter-intuitive instructions across eight categories using 1,012 Chinese-English questions curated through automated filtering and human verification.

    Source: §1 · p. 2, §2.1 · p. 3, §2.2 · p. 4

  • Results

    Across models, even simple deviations from learned paradigms trigger systematic failures; fine-tuned Qwen3-235B-A22B-Instruct and Qwen3-30B-A3B-Instruct perform poorly on the benchmark.

    Source: §3.1 · p. 7, §5 · p. 11

  • Takeaways & Limitations

    Inverse IFEval provides a diagnostic for assessing whether models follow out-of-distribution instructions that depart from conventional training paradigms.

    Source: §1 · p. 2

    The instructions are not claimed to be inherently meaningful in practice; they are intended to measure generalization in instruction following.

    Source: §2.2 · p. 4

Abstract

from arXiv · show

Large Language Models (LLMs) achieve strong performance on diverse tasks but often exhibit cognitive inertia, struggling to follow instructions that conflict with the standardized patterns learned during supervised fine-tuning (SFT). To evaluate this limitation, we propose Inverse IFEval, a benchmark that measures models Counter-intuitive Abilitytheir capacity to override training-induced biases and comply with adversarial instructions. Inverse IFEval introduces eight types of such challenges, including Question Correction, Intentional Textual Flaws, Code without Comments, and Counterfactual Answering. Using a human-in-the-loop pipeline, we construct a dataset of 1012 high-quality Chinese and English questions across 23 domains, evaluated under an optimized LLM-as-a-Judge framework. Experiments on existing leading LLMs demonstrate the necessity of our proposed Inverse IFEval benchmark. Our findings emphasize that future alignment efforts should not only pursue fluency and factual correctness but also account for adaptability under unconventional contexts. We hope that Inverse IFEval serves as both a diagnostic tool and a foundation for developing methods that mitigate cognitive inertia, reduce overfitting to narrow patterns, and ultimately enhance the instruction-following reliability of LLMs in diverse and unpredictable real-world scenarios.

1 Introduction

Idealized annotation conventions can induce cognitive inertia, making models less reliable on counterintuitive instructions. Inverse IFEval frames this as an evaluation dimension and introduces a benchmark of deliberately unconventional instructions.

1 Introduction

  • Counterintuitive instruction-following remains underexplored, despite a marked gap between model performance on conventional and counterintuitive instructions.The introduction frames the central question as whether models can unlearn stubborn training conventions to follow real instructions.Source: §1 · pp. 1–2
  • Idealized annotation conventions can give models a strong bias toward standardized responses, making instructions that deviate from training norms difficult to follow.The paper terms this difficulty cognitive inertia and links it to overfitting to post-training patterns.Source: §1 · p. 2
  • Inverse IFEval evaluates whether models can override learned conventions and follow counterintuitive instructions across eight instruction categories.The categories include question correction, intentional textual flaws, code without comments, and counterfactual answering.Source: §1 · p. 2
  • The benchmark uses a multi-stage human-in-the-loop pipeline to produce 1,012 high-quality questions across 23 domains.The pipeline combines expert seed-question design, LLM-based generation, filtering, and expert review.Source: §1 · p. 2
  • The authors identify Counter-Cognitive Ability as a critical but underexplored evaluation dimension for LLMs.The benchmark is presented as a way to examine this ability under counterintuitive instruction conditions.Source: §1 · p. 2

2 Inverse IFEval

Inverse IFEval operationalizes Counter-Cognitive Ability as following instructions that conflict with SFT conventions, then builds and evaluates a curated multilingual benchmark through human review and an optimized LLM judge.

2.1 Overview

  • Counter-Cognitive Ability tests whether models can set aside SFT paradigms and follow counterintuitive instructions, organized into eight types from Question Correction to Counterfactual Answering.Source: §2.1 · p. 3

2.2 Data collection

  • Construction reverses canonical SFT response paradigms, seeds each type with expert questions, expands via tailored LLM templates across disciplines, then filters, human-verifies, and translates the items.Source: §2.2 · p. 4

2.3 Quality Control

  • Quality control combines unanimous cross-functional seed review, length and similarity filters with cross-model checks during generation, then expert checks type fit, clarity, and rubric calibration.Source: §2.3 · p. 5

2.4 Dataset statistics

  • The benchmark contains 1,012 items across eight types and 23 domains, split evenly between Chinese and English; Code without Comments is largest at 198 items, and Computer Science is 20.2%.Counter-Conventional Formatting is smallest at 82 items; Code without Comments has the longest average reference answer, while Mid-turn Instruction Modification has the longest questions.Source: §2.4 · pp. 5–6, Table 2 · p. 6

2.5 Evaluation

  • 98% final judging accuracy, up from an 88% baseline, followed systematic optimization of judge selection, type-specific templates, and system prompts.Source: §2.5 · pp. 6–7

3 Experiments

Across two language versions, o3-high achieves the strongest Inverse IFEval performance, while further analyses examine thinking modes, instruction-type differences, and rankings against IFEval.

3.1 Main Results

  • o3-high achieves the best Inverse IFEval performance, with o3-mini and GPT-5-high following closely.Source: §3.1 · p. 7
  • Fine-tuned Qwen3-Instruct models perform poorly on a benchmark designed to test instructions that conflict with typical fine-tuning patterns.Source: §3.1 · p. 7
  • Larger models tend to perform better within the Qwen3 series.Source: §3.1 · p. 7

3.2.1 Evaluation of the thinking mechanism in LLMs

  • Qwen3 models score lower in non-thinking than thinking mode; Doubao performs best in thinking mode and worst in non-thinking mode.Source: §3.2.1 · p. 9
  • In Chinese, Doubao’s auto-thinking mode performs even worse than its non-thinking mode.Source: §3.2.1 · p. 9

3.2.2 Comparative analysis across different instruction types

  • Across eight instruction types, o3-high performs best; models score above 65 on Counterfactual Answering but struggle most with Question Correction, where half score below 30.Source: §3.2.2 · p. 9, Figure 5 · p. 10

3.2.3 A Comparative Analysis of IFEval and Inverse IFEval

  • o3-high, Gemini-2.5-Pro, and o4-mini-high rank near the top on both benchmarks, while other models show substantial ranking shifts.Source: §3.2.3 · p. 9, Figure 6 · p. 10
  • Qwen3-235B-A22B-Instruct falls from fifth on IFEval to fifteenth on Inverse IFEval, illustrating the latter benchmark’s additional instruction-following dimension.Source: §3.2.3 · p. 10

4 Related Work

Related work covers instruction-following methods and benchmarks, while noting that earlier benchmarks had not focused on inverse instructions.

Instruction Following

  • Instruction tuning has improved instruction following in language models, and related work has extended it to vision-language models.Source: §4 · pp. 10–11

Instruction Following Benchmarks

  • Prior benchmarks assess instruction following across language, vision-language, and retrieval settings, but the authors identify inverse instructions as an unaddressed focus.Source: §4 · p. 11

5 Conclusion

The conclusion presents Inverse IFEval as a benchmark for counter-intuitive and out-of-distribution instructions, finding model gaps and connecting the test to long-tail user requests.

  • Inverse IFEval evaluates LLMs on counter-intuitive and out-of-distribution instruction scenarios using eight categories of inverse instructions.Source: §5 · p. 11
  • The study reports cognitive inertia and systematic failures across models, including failures triggered by simple deviations from learned paradigms.Source: §5 · p. 11
  • The authors argue that long-tail user requests, though not always as extreme as benchmark tasks, test robustness to out-of-distribution instructions and rigid training biases.Source: §5 · p. 11

6 Contributions and Acknowledgments

The acknowledgments identify the paper’s co-first authors, co-authors, and corresponding authors.

Co-First Authors

Co-Authors

  • Yu Fu, Haojie Fan, Le Chang, Jiafan Hou, Dingling Zhang, Zhongfei Hou, Ziqiang Yang, Changxin Pu, Fei Hu, Jingkai Liu, Mengyun Liu, Yang Liu, and Xiang Gao are listed as co-authors.Source: Co-Authors · p. 12

Corresponding Authors

Appendix

Figures 7–14 demonstrate eight adversarial instruction types as tests of generalization, not claims that the instructions are inherently practical. The examples range from question correction and text flaws to counterfactual answering.

Appendix

B.1 Effects of Test-Time Compute

The paper evaluates whether more test-time sampling improves Inverse IFEval scores using Best-of-N. Scores rise as N increases, with four of five models near or above 90 at N = 32.

B.1 Effects of Test-Time Compute

  • At N = 32, four out of five models approach or surpass a score of 90.The reported scores average the English and Chinese versions.Source: Figure 15 · p. 19, §B.1 · p. 20
  • Best-of-N samples N independent outputs and selects the one with the highest score.Source: §B.1 · p. 19
  • The comparison reports average overall scores for English and Chinese at N = 1, 16, and 32.Source: §B.1 · p. 20

B.2 Comparison between English and Chinese versions

Model performance is generally similar across the English and Chinese versions, although GLM-4.5 and Qwen3-235B-A22B-Thinking perform significantly better in Chinese. The authors suggest their Chinese-community contributions may explain these exceptions.

B.2 Comparison between English and Chinese versions

  • Models that perform well in English tend to show similar success in Chinese, with scores typically close across versions.Source: §B.2 · p. 20
  • GLM-4.5 and Qwen3-235B-A22B-Thinking perform significantly better on the Chinese version than the English version.Source: §B.2 · p. 20
  • The authors suggest the two models’ Chinese-community contributions may explain their stronger Chinese-version performance.Source: §B.2 · p. 20

C Error Analysis

Error analyses show that simple instructions conflicting with training patterns can elicit failures across advanced models. The examples include formatting and question-correction errors, alongside prompts involving mid-turn modification and deliberately incorrect answers.

C Error Analysis

  • The examples show that simple instructions can fail when they contradict training patterns, with similar errors observed across different models.Source: C Error Analysis · p. 20
  • Claude-4-Opus-Thinking segmented a travel-planning response despite an instruction to avoid lists or segmented form.Source: C Error Analysis · p. 21
  • The Claude-4-Opus-Thinking examples include a Mid-turn Instruction Modification prompt.Source: C Error Analysis · p. 22
  • OpenAI-GPT-OSS-120B identified a question-correction task but selected an approximate option instead of correcting the error.Source: C Error Analysis · p. 23
  • A Doubao-1.6-Thinking example asks for only one correct answer across five questions, including arithmetic, geography, and date questions.Source: C Error Analysis · p. 24

Related papers

Loading 2509.04292v1…