PaperlayerView exact revision on arXiv

Source-linked AI summary

Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning

Jinhu Qi, Minda Hu, Wentao Zhang, Weiqiang Jin, Yanyu Chen, Junli Wang, Irwin King

arXiv:2609.00759v1cs.CL

TL;DR

Long-context in-context learning is brittle because models must extract, plan, generate, and self-verify novel rules in one forward pass. CCA compiles context into a typed intermediate representation, then uses executable verification and targeted correction; it outperforms vanilla prompting and other long-context baselines on CL-bench, with gains concentrated in structured tasks.

  • Problem

    Long-context ICL tasks require models to comply with tens to hundreds of thousands of characters of novel context, while overlooked constraints can cause whole responses to fail.

  • Method

    CCA compiles prose context into a fixed-slot typed JSON intermediate representation, enabling per-context verifier synthesis and violation-gated correction.

  • Results

    +6.0pp over Vanilla on CL-bench for Kimi K2.5, with CCA achieving the highest pass rate on every base model and outperforming ReadAgent-P and Ctx2Skill.

  • Takeaways & Limitations

    CCA supports a harness-engineering approach in which compiling structure from context can improve rubric-graded long-context performance around a frozen base LLM.

  • Takeaways & Limitations

    On LongBench-v2 fair 16K, CCA’s aggregate accuracy trails Vanilla, with benefits concentrated on multi-context reconciliation domains and disappearing on single-context tasks.

Abstract

from arXiv · show

Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output schema for a series of questions. On benchmarks that grade against every detail of the context, even strong open-weights models pass only 12-16% of tasks: a single overlooked rule fails the whole response. We argue this brittleness is structural: the dominant "read-and-reason" paradigm asks the model to extract, plan, generate, and self-verify in one forward pass. We therefore ask whether explicit context compilation can fix it, how it compares to existing long-context strategies (gist retrieval, multi-agent self-play), and where the resulting harness benefit holds across task structure and model scale. We propose the Context Compilation Architecture (CCA), whose central novelty is a typed intermediate representation (IR) with fixed slots (rules.{must_do, must_not, conditional}, output_spec, available_tools, data_profile) into which any prose context is compiled once; executable verifiers and a violation-gated correction loop follow as downstream consequences. On CL-bench (1,899 tasks across 4 open base models), CCA outperforms vanilla prompting and two long-context baselines (ReadAgent-P, Ctx2Skill) on every base model, lifting Kimi K2.5 from 15.4% to 21.4% with gains concentrated on rule-dense sub-categories. Code and cached completions are available at https://github.com/TonyQJH/cca-emnlp2026.

1 Introduction

Long-context ICL tasks require models to satisfy many detailed contextual constraints, but vanilla read-and-reason prompting often misses individual rules. CCA addresses this brittleness by compiling context into typed structure and reusable verification artifacts before answering.

  • CL-bench tasks require satisfying 5–20 rubric criteria, and missing one constraint marks the task failed.
  • 12–16%: strong open base models pass only this share of rubric-graded tasks under vanilla prompting.
  • The proposed explanation is structural: extraction, planning, generation, and self-verification compete within one forward pass as constraints multiply.
  • CCA compiles prose into a fixed-slot typed JSON IR containing rules, knowledge, workflow, and output schema.
  • CCA follows compilation with executable per-context verifiers and violation-gated correction, replacing diffuse self-judgment with mechanical checking.

2 Related Work

Related long-context methods compress or retrieve prose, generate skills, or use code for computation; CCA instead compiles each context into typed structure and reusable verification code.

  • ReadAgent-P paginates contexts into gists and retrieves relevant pages, but gist compression can omit verbatim facts required by rubric grading.
  • Ctx2Skill uses a Challenger/Reasoner/Judge/Proposer/Generator self-play loop to build a natural-language skill library retrieved at inference.
  • Existing code-as-reasoning methods generate programs per question to produce answers or delegate computation to an interpreter.
  • CCA generates per-context code once to verify whether a natural-language answer satisfies typed contextual rules, rather than producing the answer itself.
  • CCA differs from DSPy by compiling each given context into a typed IR and per-context verifiers instead of optimizing a fixed LM-call pipeline.

3 Context Compilation Architecture (CCA)

CCA separates context compilation from per-task answering: it builds reusable typed artifacts and verifiers once, then drafts and conditionally corrects each answer using them.

  • Pipeline structure: CCA processes one context group through four stages, with Compiler and CodeGen running once per context and Reasoner, execution, and correction running per task.
  • Pipeline structure: The pipeline factorizes into a per-context compilation phase and a per-task output phase that reuses the compiled artifacts.
  • Pipeline structure: The Compiler emits typed JSON IR, CodeGen selects verifier modules, and the Reasoner-1 draft is checked by rule and format verifiers.
  • Verification and correction: Correction fires when at least 2 violations are found, using the violation list and original draft to make minimal local edits.
  • Pipeline composition: Equation 1 identifies amortized context artifacts, direct IR checklist injection, cached data summaries, and draft-time verifier operation.
  • Verification and correction: The IR enters Reasoner-1 as a checklist, while a data-analyzer summary can enter separately as code-execution results.
  • Compiler: The Compiler preserves counterfactual content, extracts must/should/always/never rules, records exact terms, and marks rules for mechanical checking.
  • Code generation: The dispatcher emits rule_checker for codeable rules, format_validator for sufficiently complex formatting requirements, and data_analyzer for tabular data.

4 Experiments and Results

On CL-bench, CCA is evaluated against direct prompting and two long-context strategies across four open models, with gains strongest for larger models and rule-dense tasks. The evaluation also examines cost, amortization, and cross-benchmark behavior.

  • Evaluation setup: CL-bench contains 1,899 tasks across four domains, with long contexts and 5–20 independently graded rubric criteria; a task passes only when every criterion is satisfied.The benchmark includes contexts up to 247K characters and uses must-include, must-not-include, and formatting criteria.
  • Evaluation setup: CCA is compared with Vanilla direct prompting, ReadAgent-P gist retrieval, and Ctx2Skill multi-agent self-play under fixed-temperature evaluation.All four open models receive identical prompts, while GPT-5.1 supplies per-criterion and overall grading through the official script.
  • RQ1 — effectiveness: CCA achieves the highest Overall pass rate on every base model, including lifts of +6.0pp on Kimi K2.5, +5.1pp on GLM-5, and +2.7pp on DeepSeek-V3.2.The corresponding rates are 15.4→21.4%, 16.1→21.2%, and 15.0→17.7%; all three improvements are significant by paired McNemar tests.
  • RQ3 — moderators: On Qwen3-Next-80B, CCA’s lift is only +0.5pp and is not significant, indicating a model-capacity boundary for consuming the structured supplementary signal.The reported test is z = 0.38, p = 0.70.
  • RQ2 — comparisons: Neither ReadAgent-P nor Ctx2Skill matches CCA’s Overall score on any base model, while Ctx2Skill costs approximately 5.7× more tokens per task than CCA.Figure 2 places CCA on the upper-middle Pareto frontier and shows Ctx2Skill near Vanilla’s pass rate despite its higher cost.
  • RQ3 — moderators: CCA’s lift concentrates in rule-dense PTE and RSA domains, remains positive but smaller in DKR, and does not win on any model in open-ended EDS.CCA wins all four PTE columns and three of four RSA columns.
  • Cross-benchmark probe: On LongBench-v2, CCA scores 53.88% versus Vanilla’s 57.85% overall, but gains +5.69pp on Multi-Doc QA and +10.25pp on Long-Dialog.The cross-benchmark probe uses 503 multi-choice tasks with matched max_tokens=16,384.
  • Cost: CCA separates per-context offline compilation from per-task inference, amortizes 6.4K offline tokens over 5.13 Kimi tasks per context, and runs deterministic verifiers at zero LLM tokens.On Kimi K2.5, online inference costs 27.7K tokens per task and wall-clock time is approximately 1.6× Vanilla; CCA-V2 gives +4.72pp at 1.90× tokens.

5 Discussion

The discussion finds that CCA’s gains arise primarily from preserving rubric-relevant structure in a typed IR, while its benefit varies with task structure and model capacity. It also shows that CCA outperforms lossy long-context baselines but can add overhead or underperform when structure is less useful.

  • Answering RQ1: Effectiveness of Context Compilation: CCA’s IR-as-checklist injection is the load-bearing mechanism in the ablation.Removing F2 drops performance 3.07pp, versus 1.28pp for Reasoner-2 correction.
  • Answering RQ1: Effectiveness of Context Compilation: The four ablation contributions sum exactly to CCA’s +6.00pp Full-versus-Vanilla gap.The decomposition assigns +1.82pp to the scaffold, +2.37pp to F2, +0.53pp to verifier execution, and +1.28pp to compile-then-verify.
  • Answering RQ2: Why Existing Strategies Don’t Improve: ReadAgent-P and Ctx2Skill lose rubric-critical verbatim signals through gisting or skill distillation, whereas CCA preserves them in its IR and executable checks.CCA beats both baselines on every model-and-overall cell and reaches the only large positive lift on the reported Pareto frontier.
  • Answering RQ3: Where the Compilation Harness Holds: The largest gains occur on structurally dense tasks, including Legal & Regulatory at +20 to +27pp, Management at +11 to +25pp, and Workflow Orchestration at +6 to +11pp.These categories expose rules, decision frameworks, exact terms, tools, or workflow steps that the IR captures explicitly.
  • Answering RQ3: Where the Compilation Harness Holds: CCA’s lift is bounded by model capacity: Qwen3-Next-80B gains only +0.5pp, which is not significant (p=0.70).The paper attributes this pattern to difficulty integrating the structured IR and verifier results with the original context.
  • When to Use CCA: A Domain-Selective Harness: CCA is domain-selective: it underperforms Vanilla on LongBench-v2 overall, while CCA-Adaptive raises aggregate performance to 60.24% (+2.39pp versus Vanilla).The adaptive router applies CCA when verifier modules are emitted and falls back to Vanilla otherwise.

6 Conclusion

CCA compiles prose context into a typed, machine-checkable IR, with verifier synthesis and violation-gated correction following from that representation. It achieves the highest CL-bench pass rate on every base model tested, while its gains are smaller on the capacity-bounded Qwen3 case and it also outperforms the two long-context baselines.

  • Conclusion: CCA’s central contribution is a typed IR with fixed slots that converts prose context into a machine-checkable data structure.Verifier synthesis and violation-gated correction are downstream consequences of this representational contract.
  • Conclusion: CCA achieves the highest CL-bench pass rate on every base model, with gains of +6.0pp, +5.1pp, and +2.7pp over Vanilla on three models.The gains are significant for Kimi K2.5, GLM-5, and DeepSeek-V3.2; Qwen3-Next-80B shows a smaller, non-significant +0.5pp.
  • Conclusion: CCA also outperforms ReadAgent-P and Ctx2Skill, while CCA-Adaptive reaches 60.24% on LongBench-v2, +2.39pp over Vanilla.The adaptive result uses a cca_meta.ir_ok-gated router.

7 Limitations

The paper identifies scope and evaluation boundaries for CCA, including limited ablation coverage, incomplete schema coverage, compiler robustness, domain selectivity, and single-judge scoring.

  • Scaling the ablation: Component-level ablation is reported only on Kimi K2.5, leaving the model-capacity boundary for harness benefits less sharply characterized.Repeating the isolation on the other three base models, especially at smaller activation, is identified as future work.
  • Schema coverage: The current IR covers rule-heavy, procedural, and data-analysis contexts, while spatial and temporal structures remain direct extension targets.Examples include geometric reasoning, layout, long-horizon planning, and event ordering.
  • Compiler robustness: Unparseable IRs occurred for < 1% of context groups and fell back to direct prompting, but formal safety-critical verification remains future work.The reported fallback prevents malformed IRs from blocking evaluation, while the stronger formal guarantee is not yet established.
  • Domain-selective on open-ended long-context: On LongBench-v2 fair 16K, CCA trails Vanilla overall, with gains concentrated on multi-context reconciliation and absent on single-context tasks.CCA scores 53.88% versus Vanilla’s 57.85%; Multi-Doc QA and Long-Dialog gain +5.69pp and +10.25pp respectively.
  • Single-judge scoring: CL-bench pass rates rely on a single GPT-5.1 judge, although cross-judge checks find it strictest and a full human audit remains future work.Cross-judge triangulation on a stratified 500-task subset reports pairwise κ of 0.40–0.61.

A Implementation Reference: Prompt Cores and IR Schema

The appendix specifies CCA’s prompt cores, typed IR schema, verifier behavior, reasoning priorities, correction protocol, truncation policy, and ablation feature definitions.

  • A Implementation Reference: Prompt Cores and IR Schema: The implementation appendix exposes design-critical prompt instructions and the Compiler’s IR schema while omitting boilerplate formatting and politeness.Full prompts and code are released with supplementary material.
  • A.1 Compiler Prompt — Essential Sections: The Compiler preserves fictional or counterfactual context verbatim and extracts exact terms that downstream rubrics may require literally.Exact terms include role identifiers, tool names, status codes, identifiers, rating tiers, and required phrases.
  • A.2 IR Schema: The IR records rules, exact terms, output specifications, and codeability metadata, including source quotes and optional verification hints.Rules are grouped by polarity and may be mechanically checked by CodeGen; exact terms include categories and example uses.
  • A.3 CodeGen Prompt Cores: CodeGen produces rule, format, and data modules designed to avoid false positives and to parse malformed data defensively without raising.Format checks use appropriate validators, while data analysis tries multiple delimiters and returns partial results on failure.
  • A.4 Reasoner-1 System Prompt — Essential Sections: Reasoner-1 treats the checklist and original context as authoritative, prioritizing specific contextual rules over general principles or prior knowledge.This conflict-resolution layer is intended for overlapping rules, including in legal and management contexts.
  • A.5 Reasoner-2 Correction Prompt: The Reasoner-2 correction prompt supplies codeable and format violations, requiring precise flagged fixes while preserving the draft’s other structure and content.Violations and drafts are truncated to fit the prompt budget, and only flagged edits support monotone correction.
  • A.6 Head/Tail Context Truncation (F7): When context exceeds the Reasoner budget, F7 retains approximately 70% of the head and 30% of the tail around a truncation marker.The policy preserves front-loaded rules and the latest user question or turn.
  • A.7 F-Code Definitions for the Ablation: CCA’s ablation units F1–F7 span compilation, IR injection, CodeGen, verification, data analysis, correction, and head/tail truncation.The pseudocode separates reusable per-context artifacts from per-task drafting and correction.

C Full Result Matrix and Statistical Tests

The full result matrix shows that CCA’s gains are strongest in rule-dense CL-bench domains and significant for the three larger models, while qualitative cases illustrate how compilation and correction address missed constraints.

  • Full result matrix: Table 5 reports pass rates across four methods, four base models, and four domains, with Overall rows aggregating all 1,899 tasks.The appendix matrix matches the corresponding main-table Overall results.
  • Full result matrix: Rule-dense sub-categories show consistently positive CCA lifts across models, whereas open-ended sub-categories show consistent reversals.This pattern supplies the empirical basis for task-structural-density moderation.
  • Statistical tests: McNemar testing finds significant CCA-versus-Vanilla differences for the three larger models, but only a +0.5pp lift consistent with the null for Qwen3-Next-80B.The paper uses this pattern as evidence for a model-capacity moderator.
  • Qualitative mechanism cases: On a coordinator task, CCA extracted 24 rules and five expert-type exact terms, then corrected two verifier-identified violations into an accepted answer.Baselines respectively missed a critical tier, timing detail, or expert-role distinction.
  • Qualitative mechanism cases: On a directed-multigraph coding task, CCA extracted node-label exact terms and corrected three verifier-flagged violations, while baselines altered labels, omitted arrows, or used overly long comments.The corrected CCA output is marked cca_meta.corrected = True.
  • Qualitative mechanism cases: On a SEV2 orchestration task, CCA encoded 64 rules, including mandatory tool calls and a prohibition on switching modes, then corrected four violations.The correction fired despite the user’s override request, producing cca_meta.corrected = True.

D.4 EDS / Experimental Data — proportionality constant with ASCII-only constraint

The EDS case tests whether CCA preserves a proportionality-constant answer under a five-criterion rubric, including ASCII-only formatting. CCA combines compiled rules, data analysis, and verification to address failures seen in the baselines.

  • Task and rubric: The EDS rubric requires a numeric value, units, uncertainty estimate, matching uncertainty units, and ASCII-only characters.A response failing any criterion is unsuccessful.
  • Baseline failures: Vanilla produced the correct number and units but failed because its typeset multiplication sign was non-ASCII.Its reported value was 1.27 × 10−7 W m−2 K−5 ± 0.15 × 10−7.
  • Baseline failures: ReadAgent-P reported 2.3±0.15 without units, violating the unit and matching-unit criteria.The gist-based baseline omitted required dimensional information.
  • Baseline failures: Ctx2Skill refused the task because it found no repeated measurements, producing no numerical value and violating the first rubric criterion.Its self-play skill library did not yield a rubric-compliant answer in this case.
  • CCA behavior: CCA extracted 11 rules, generated a data_analyzer module, pre-computed a power-law fit, and injected both the result and ASCII rule into Reasoner-1.Reasoner-1 used pure ASCII notation, reporting 1.18e-7 rather than a typeset multiplication sign.
  • CCA behavior: The verifier flagged at least two violations and triggered Reasoner-2, illustrating the correction loop on context-dense tasks.The case used CCA’s additional non-LLM compute path, data_analyzer(raw_data).

G.1 Per-domain pass rates of the ablation variants

The ablation results show that CCA’s gains are concentrated in rule-dense domains, especially when the IR checklist is added, while open-ended EDS remains comparatively flat. The correction loop adds further lift but at substantially higher token cost.

  • Per-domain pass rates: Adding F2 changes RSA from 16.6% to 18.0% and PTE from 20.6% to 24.2% on Kimi K2.5.These are the largest domain-specific swings in the ablation variants.
  • Component contributions: The F2 IR-as-checklist injection is the dominant single mechanism, while the correction loop is the second-largest contributor.Verifier execution alone mainly logs violations and is not itself the principal source of improvement.
  • Component contributions: With F2 enabled, F4+F6 contribute +1.81pp, versus +1.11pp with F2 disabled, yielding a +0.70pp positive interaction.The checklist makes the correction loop’s violation list more actionable.
  • Cost-benefit trade-off: F2 adds about 2K tokens for +2.37pp, whereas F6 adds about 10–11K tokens for +1.28pp.F6 is roughly an order of magnitude less token-efficient per percentage point of lift.
  • Per-domain pass rates: EDS is comparatively flat across V6, V3, V2, and V4, with Full CCA at 11.6% below Vanilla at 12.1%.This supports limited or negative lift in the open-ended domain.

H.1 All four methods as offline-prep + online-infer architectures

All four methods separate context-level preparation from task-level inference, but their cached artifacts differ substantially. CCA spends more online tokens than simpler baselines yet converts that cost into a large pass-rate lift, unlike the other long-context strategy.

  • Architectural split: ReadAgent-P prepares paginated gists offline and retrieves pages online, while Ctx2Skill prepares a shared skill library through offline self-play.Ctx2Skill is the closest architectural analogue to CCA because both cache context-derived artifacts.
  • Cost-quality comparison: CCA spends about 3× Vanilla’s and 2.5× ReadAgent-P’s per-task tokens, while using about 18% of Ctx2Skill’s tokens.This is the reported cost-quality positioning on Kimi K2.5.
  • Cost-quality comparison: Ctx2Skill’s offline phase costs about 178K amortized tokens per task versus CCA’s about 6.4K, roughly a 30× difference.Despite the heavier offline investment, Ctx2Skill contributes only +0.1pp, while CCA contributes +6.00pp.
  • Cached artifacts: CCA’s cached typed IR and executable checks are contrasted with Ctx2Skill’s natural-language skills, which the paper characterizes as a lossy abstraction.The comparison emphasizes what the offline phase produces, not merely its token expenditure.
  • Cached artifacts: CCA’s online cost is 27.7K tokens per task after amortizing its 6.4K offline subtotal, with Reasoner-2 contributing an average 11.2K tokens.Reasoner-2 fires on approximately 62% of tasks.
  • Architectural split: CCA runs Compiler and CodeGen once per context, then runs reasoning, verification, and correction once per downstream task.Its two-phase structure follows the factorization in Equation 1 and amortizes offline work across shared tasks.
  • Deployment choices: CCA-V2 recovers 4.72 of the 6.00pp lift while saving about 51% of Full CCA’s marginal tokens over Vanilla.Full CCA is presented as the maximum-quality endpoint, while V2 targets latency- or cost-sensitive serving.

I Temperature Robustness

CCA’s advantage persists under non-greedy decoding, although sampling reduces its absolute pass rate. The matched temperature-0 comparison isolates the main result from sampling differences, while the temperature-1 run provides a conservative robustness check.

  • Robustness result: At temperature 1, Full CCA still beats temperature-0 Vanilla by +3.35pp on Kimi K2.5.The re-run used the full 1,899-task CL-bench set with other settings held constant.
  • Sampling effect: Temperature 0 gives a 2.65pp higher absolute pass rate than temperature 1 on the same Full CCA pipeline.The paper associates this difference with sampling noise under all-criteria rubric grading.
  • Interpretation: The main +6.00pp Full-versus-Vanilla gap is measured under matched temperature-0 conditions, while temperature 1 retains a win despite asymmetric sampling noise.The authors characterize the temperature-1 result as a conservative lower bound on CCA’s structural advantage.
Loading 2609.00759v1…