PaperlayerView exact revision on arXiv

Source-linked AI summary

Schema-Key Wording as an Instruction Channel in Structured Generation under Constrained Decoding

Yifan Le

arXiv:2604.14862v3cs.CLcs.AI

TL;DR

Existing work largely treats schemas as structural constraints, leaving schema-key wording as a less-studied potential instruction signal. The paper studies prompt and schema-key instruction channels with controlled comparisons and projection-aware analysis, finding heterogeneous accuracy effects and non-additive interactions, especially with stronger evidence on GSM8K.

  • Problem

    Prior work mainly treats schemas as structural constraints, leaving limited study of whether schema-key wording can guide generation as an instruction signal.

    Source: §1 · p. 1

  • Method

    The paper compares instruction signals placed in prompts, schema keys, or both under controlled conditions and analyzes when key advantages persist after grammar projection.

    Source: §3.2 · p. 3, §3.4 · p. 5

  • Results

    Schema-key wording affects accuracy across GSM8K and Math500, with model- and task-dependent effects, non-additive channel interactions, and substantially stronger evidence on GSM8K.

    Source: §4.5 · p. 7, §5 · p. 8

  • Takeaways & Limitations

    Schema wording should be treated as part of instruction design, and fair evaluation and reproducible deployment should report prompts, schemas, key names, decoding backends, and parsing rules together.

    Source: §5 · p. 8

    The study covers mathematical reasoning benchmarks and several open-source Qwen and Llama-family models, so its observed sensitivities may not transfer to other tasks or model families.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Constrained decoding is widely used to make large language models produce structured outputs that satisfy schemas such as JSON. Existing work mainly treats schemas as structural constraints, overlooking that schema-key tokens also enter the autoregressive context and may guide generation. To the best of our knowledge, we present the first systematic study of schema keys as an implicit instruction channel under constrained decoding. We formulate structured generation as a multi-channel instruction problem, where task signals can be placed in prompts, schema keys, or both. We further provide a projection-aware analysis that gives a sufficient condition under which an unconstrained expected-score advantage of an instructional key is preserved after grammar projection. Experiments on GSM8K and Math500 across seven language models show that changing only schema-key wording can substantially affect accuracy, with both positive and negative effects across models. Prompt-level and schema-level instructions also interact non-additively. The evidence is substantially stronger on GSM8K than on Math500. Our findings show that schema design is not merely output formatting, but part of instruction specification in structured generation.

1 Introduction

Structured generation ensures valid formats, but validity alone does not address how instructions shape task performance. This section frames schema-key wording as an instruction channel and introduces controlled comparisons and a projection-aware analysis.

1 Introduction

  • Existing constrained-decoding work emphasizes structural validity, while structured generation also warrants study as an instruction-design problem.The introduction distinguishes valid output structure from task success.Source: §1 · p. 1
  • Schema keys are generated prefixes that can cue the following value, making their wording an implicit instruction channel without changing canonical output structure.The paper presents this as the first systematic study of schema keys in this role.Source: §1 · pp. 1–2
  • A multi-channel formulation compares instruction in the prompt, schema keys, or both against no additional instruction while holding other generation factors fixed.The four settings isolate instruction placement.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • A projection-aware analysis gives a sufficient condition for an instructional key’s unconstrained expected-score advantage to persist after grammar projection.The analysis offers a distribution-level perspective on why effects may vary across models and settings.Source: §1 · pp. 1–2
  • Across GSM8K and Math500, key wording affects accuracy, effects vary by model and task, prompt and schema instructions interact non-additively, and evidence is stronger on GSM8K.The reported interaction effects can be complementary, redundant, or conflicting depending on the model.Source: §1 · p. 2

2 Related Work

Related work has primarily developed ways to enforce structural constraints and studied how constraints affect model behavior. The paper distinguishes its focus on whether schema-key wording itself acts as an instruction signal.

2.1 Structured Generation with Constrained Decoding

  • Constrained decoding restricts candidate tokens using a schema, and prior methods include CFGs, PDAs, FSMs, parsing, extra generation stages, and postprocessing.These approaches primarily address structural validity, efficiency, or correctness.Source: §2.1 · pp. 2–3

2.2 Effects of Structural Constraints on Model Behavior

  • Prior studies examine constraint-related distributional bias and changes in diversity, creativity, or reasoning, with effects varying by model scale, task, and constraint design.The related work also notes that grammar constraints can improve performance in some tasks.Source: §2.2 · p. 3
  • This paper instead examines whether schema-key wording itself acts as an implicit instruction signal under constrained decoding.Source: §2.2 · p. 3

3 Method

The method treats schema keys as generated prefixes that can condition subsequent values, while preserving the canonical output structure. It analyzes how instruction placement and grammar projection relate to task performance using controlled comparisons.

3.1 Problem Setup

  • Schema keys precede their values and condition subsequent generation, so changing only a target key’s wording can alter generated context while preserving the same canonical interpretation.The intervention holds field order, value types, and number of fields fixed.Source: §3.1 · p. 3

3.2 Projection-Aware View of Schema-Key Instructions

  • A sufficient condition preserves an instructional key’s unconstrained expected-score advantage after grammar projection, but it is not necessary.The analysis is distributional; under greedy decoding, the bound is diagnostic rather than a deterministic guarantee for every generated trajectory.Source: §3.2 · p. 4

3.3 Structured Generation as Multi-Channel Instruction

  • The multi-channel framework separates instruction signals in prompts from signals in decoder-enforced schema keys, testing no instruction, key-only, prompt-only, and both-channel settings.These settings distinguish instructions described to the model from those forced into generated context or supplied through both routes.Source: §3.3 · pp. 4–5

3.4 Controlled Comparison and Effect Measures

  • The controlled comparison holds the model, dataset, decoding setup, and canonical output properties fixed, varying only instruction placement and measuring channel effects and their interaction.Positive interaction values indicate synergy, near-zero values approximate additivity, and negative values indicate redundancy or conflict.Source: §3.4 · p. 5

4 Experiments

Experiments across seven models show that schema-key wording can change accuracy under constrained decoding, with effects varying by model and task. Prompt- and schema-level instructions can interact non-additively, and the evidence is stronger on GSM8K than Math500.

4.1 Experimental Setup

  • Experiments report answer accuracy on GSM8K and Math500 across seven named models, comparing None, Key-only, Prompt-only, and Both under a fixed JSON structure, parsing, decoding backend, and inference configuration.Unadjusted 95% paired item-level percentile bootstrap confidence intervals use 20,000 resamples.Source: §4.1 · pp. 5–6

4.2 Main Results under Different Instruction Placements

  • On GSM8K, Key-only raises Qwen2.5-7B from 81.27 to 89.08, while schema-key effects vary across models and prompt-plus-key performance is not uniformly best.DeepSeek-R1-Distill-Qwen-7B performs best under Prompt-only, while Both is best for several other models.Source: §4.2 · pp. 6–7

4.3 Cross-Model Sensitivity to Instruction Channels

  • Representative GSM8K cases show Qwen2.5-7B benefits most from schema-level instruction, Qwen2.5-Coder-14B from using both channels, and Llama-3.2-3B is harmed by Key-only.The examples illustrate model-level variation, not a family-level trend.Source: Table 3 · p. 6, §4.3 · p. 7

4.4 Single-Channel and Interaction Effects

  • On GSM8K, key effects are supported for all seven models, prompt effects for six, and interaction effects for six; Math500 prompt effects are statistically inconclusive.GSM8K key effects are positive for five models and negative for two; selected Math500 key and interaction effects are supported.Source: §4.4 · p. 7, Figure 3 · p. 8

4.5 Summary of Findings

  • The results support treating schema keys as instruction-bearing design choices, because wording can affect accuracy even when the structure and decoding engine remain fixed.The reported effects vary across models and tasks, and prompt- and schema-level instructions can interact positively or negatively.Source: §4.2 · p. 6, §4.5 · p. 7

5 Discussion and Conclusion

The discussion explains that schema keys enter the autoregressive context and can affect downstream accuracy even when structural and decoding choices are fixed. The authors argue that schema wording belongs in instruction design and should be reported alongside prompts and other generation settings.

5 Discussion and Conclusion

  • Schema keys are generated as part of the autoregressive context, so their wording can alter downstream accuracy without changing schema structure, field types, parsing rules, or decoding backend.Prompt-side descriptions and decoder-enforced keys supply instructions through different routes, and their combination can yield heterogeneous, non-additive effects.Source: §5 · p. 8
  • For fair evaluation and reproducible deployment, report prompts, schemas, key names, decoding backends, and parsing rules together.The authors frame schema wording as part of instruction design rather than an arbitrary implementation choice.Source: §5 · p. 8

Limitations

The study is bounded to mathematical reasoning benchmarks and a narrow schema design, with stronger statistical evidence on GSM8K than Math500. Its evaluated models and schema variants also limit how broadly the findings can be applied.

Limitations

  • Evaluation covers several open-source Qwen and Llama-family models and a Qwen-based distilled reasoning model, so transfer to other families or proprietary models is uncertain.Source: Limitations · p. 9
  • Field descriptions, field order, nesting depth, optional fields, enum values, and natural-language schema documentation remain additional schema factors for study.Source: Limitations · p. 9
  • The evidence is substantially stronger on GSM8K than Math500, where many effects remain inconclusive.The experiments focus on mathematical reasoning benchmarks, not the full range of structured-generation scenarios.Source: Limitations · p. 9
  • The tested schema is structurally simple, with two required fields; nested schemas, optional fields, and a broader range of key paraphrases were not tested.Source: Limitations · p. 9

Ethics Statement

The study uses public benchmarks and open-source models without private user data, human subjects, or personally identifiable information. It emphasizes transparent reporting of schema and decoding choices because key wording can influence performance.

Ethics Statement

  • The study uses publicly available benchmarks and open-source models, without private user data, human subjects, or personally identifiable information.Source: Ethics Statement · p. 9
  • Because schema-key wording can influence model performance, structured-generation evaluations should report schemas, key names, decoding backends, and parsing rules alongside prompts and models.Source: Ethics Statement · p. 9
  • Schema variants should be evaluated carefully in high-stakes deployments, where field-name changes may affect correctness, robustness, or user trust.Source: Ethics Statement · p. 9

A Derivation of the Projection-Aware Schema-Key View

The appendix develops a distributional account of constrained decoding and schema-key conditioning. It treats grammar enforcement as a projection onto valid continuations and describes how key wording can condition the value distribution.

A Derivation of the Projection-Aware Schema-Key View

A.1 Constrained Decoding as KL Projection

  • Constrained decoding is characterized as a reverse-KL projection of the model distribution onto valid next-token distributions.Source: §A.1 · p. 10
  • At sequence level, the analysis defines a cumulative projection tax under matched stopping conventions and support assumptions; the tax is larger when decoding must strongly override the model’s preferred distribution.Source: §A.1 · pp. 10–11

A.2 Schema-Key Conditioning

  • Because key tokens precede their values, key wording conditions the continuation distribution; the conditional formulation isolates this semantic effect from the structural cost of forcing key tokens.Source: §A.2 · p. 11

A.3 Bounding the Effect of Projection

This subsection bounds the effect of projection by relating differences in a bounded metric across distributions to total variation and then to a bound involving Bk(x).

A.3 Bounding the Effect of Projection

  • For the bounded metric over two continuation distributions, the analysis relates the difference to total variation and then applies Pinsker’s inequality to obtain a bound involving Bk(x).Source: §A.3 · p. 11

A.4 A Sufficient Condition for CoT-Key Improvement

The analysis gives a sufficient condition for an instructional key’s unconstrained expected-score advantage to survive grammar projection, and offers an optional activation-based interpretation of the semantic gain.

A.4 A Sufficient Condition for CoT-Key Improvement

  • The sufficient condition guarantees that an instructional key’s unconstrained expected-score advantage is preserved after grammar projection, but it is not necessary.The condition may fail when the model is insensitive to key wording or the CoT-style key incurs a large projection tax.Source: §A.4 · p. 11
  • The condition is unlikely to hold when the model is insensitive to key wording, and may fail if the CoT-style key introduces a large projection tax.These cases offer one possible explanation for variation in key-level effects across models.Source: §A.4 · p. 11

A.5 An Optional Activation-Based Interpretation

  • The optional activation-based interpretation links the sufficient condition to useful reasoning continuations: if the CoT-style key increases A_k while the conditional quality gap remains positive, the semantic gain is expected to increase.This interpretation is not a theorem; it explains how key wording can change the semantic benefit term in Equation 7.Source: §A.5 · p. 11

B Decoding-Regime Sensitivity

Under stochastic decoding at temperature 0.8, several large GSM8K effects remain stable across runs, although selected effects are decoding-regime-sensitive. The sensitivity analysis repeats the factorial evaluation across three runs and reports run-level variation and bootstrap intervals.

B Decoding-Regime Sensitivity

  • The sensitivity analysis repeats the full factorial evaluation three times at temperature=0.8 while keeping models, datasets, prompts, schemas, backend, and other inference settings fixed.The reported mean effects use the three runs.Source: B Decoding-Regime Sensitivity · p. 12
  • The tables report mean factorial effects, run-level standard deviations, and unadjusted 95% paired item-level percentile bootstrap confidence intervals computed from run-averaged item-level contrasts.The reported measures are in percentage points.Source: B Decoding-Regime Sensitivity · p. 12, Table 4 · p. 13, Table 5 · p. 13
  • Several large GSM8K effects persist across runs: Qwen2.5-7B retains a positive schema-key effect, while Llama-3.2-3B retains a large negative effect and positive interaction; selected effects, especially for DeepSeek-R1-Distill-Qwen-7B, are decoding-regime-sensitive.The GSM8K and Math500 tables report factorial effects under stochastic decoding at temperature=0.8 across three runs.Source: B Decoding-Regime Sensitivity · p. 12, Table 4 · p. 13, Table 5 · p. 13
Loading 2604.14862v3…