Source-linked AI summary
Instruction Duplication as an Inference-Time Control Primitive
Victor Lavrenko
TL;DR
The paper asks whether procedural instruction following can be controlled in ways useful to systems that inspect or repair generated trajectories. It introduces instruction duplication as a black-box inference-time control and finds that it improves machine-addressable trajectory state, with downstream value depending on the consuming controller.
Problem
Downstream verifiers and controllers may require explicit intermediate states, not only correct final answers, but procedural state and task correctness can diverge.
Method
The study evaluates instruction duplication across models, medical benchmarks, placement conditions, and scheduled generations, measuring protocol completion, accuracy, perceptual changes, and downstream AE outcomes.
Results
In the headline downstream result, the SSNHL endpoint rose from 84.2% with reproduced system-only AE to 97.1% with a trailing duplicate, while conductive branch preservation fell from 78.6% to 73.8%.
Takeaways & Limitations
Instruction duplication is a placement-sensitive control whose practical value can emerge when a downstream system monitors or edits the exposed trajectory.
Takeaways & Limitations
The selected models and medical multiple-choice datasets do not establish transfer to other domains, instruction families, or serving stacks.
Abstract
from arXiv · showhide
Procedural instruction following is a basic requirement for controllable language-model systems, especially when generated trajectories are inspected or repaired downstream. We introduce instruction duplication, a minimal black-box inference-time control that repeats only the procedural instruction, without retraining or decoding changes. Across seven instruction-tuned models, 300 medical multiple-choice questions, eight placement conditions, and 16,800 scheduled generations, moving from one to two copies raises the deterministic All-8 diagnostic--responses passing all eight observable tests--from 90.22% to 93.17% (+2.95 percentage points), eliminating 30.2% of the failures remaining after one copy. Pre-provisional TF-IDF recall rises from 73.44% to 74.81% (+1.38 points; Holm-adjusted p < .001), while final-answer accuracy remains exactly 60.21%. Premature commitment increases from 1.52% to 2.30% (p_Holm = .00536). A blinded challenge audit yields 10/30 directional confirmations, 20/30 perceptual ties, and no reversals; its prespecified 28/30 confirmation criterion is not met. Yet this distinction can matter operationally when a downstream system acts on the generated trajectory. In Answer Engineering (AE), where explicit trajectory state determines local repair, the published reason-first no-editing SSNHL endpoint was 25.1%; system-only AE was later reproduced at 84.2%, and the same trailing duplicate raised it to 97.1%. For conductive diagnostic branch preservation, the corresponding values are 58.9% published without editing, 78.6% with reproduced AE, and 73.8% with AE plus duplication--a within-AE decrease, but still 14.9 points above the no-editing baseline. Instruction duplication is therefore a low-complexity, placement-sensitive control whose practical value can emerge through the downstream system that consumes the exposed trajectory.
Introduction
Language-model outputs may need to expose procedural states, not just final answers, so downstream verifiers and controllers can inspect or repair trajectories. This paper asks whether repeating only the procedural instruction changes that machine-addressable state without duplicating the query.
- Motivation: Downstream systems may require explicit evidence inventories, comparisons, delayed commitments, or reconsiderations as machine-addressable trajectory states.A correct final answer can still omit a state needed by a verifier, auditor, or runtime controller.
- Research question: Prior work motivates repetition and placement as inference-time interventions, but this study isolates repetition of the procedural instruction while leaving the substantive query unchanged.The narrower question is whether procedural-instruction duplication changes the explicit state available to downstream systems.
- Intervention: Instruction duplication adds a second copy of the same procedural instruction without adding rules, examples, rewards, verifiers, or model access.Unlike whole-prompt repetition, the query is not duplicated; the intervention targets procedure execution rather than task-content re-exposure.
- Contributions: The paper contributes a minimal black-box control, a 2 × 2 × 2 placement factorial, and evidence that explicit protocol state can change while aggregate answer accuracy does not.The blinded audit and downstream Answer Engineering experiment motivate the operational relevance of this dissociation.
Related Work
Related work frames instruction duplication against parameter updates, process supervision, prompting, constrained decoding, diagnostic reasoning, and trajectory editing. The paper positions its contribution as an inference-time intervention targeting observable procedural state.
- Training and inference-time control: Instruction tuning and preference optimization alter model parameters, whereas process supervision supplies learning signals over intermediate steps.These approaches differ from inference-time controls that require no retraining.
- Training and inference-time control: Chain-of-thought prompting and decomposition can change visible trajectories without retraining, although visible rationales need not faithfully reveal internal answer causes.Grammar-constrained decoding is another more direct control mechanism discussed in the related work.
- Diagnostic reasoning: Clinical diagnostic reasoning motivates the protocol’s contrastive structure: generate hypotheses, compare them with findings, identify discriminating evidence, and revise surviving hypotheses.The paper relates this structure to iterative hypothesis testing and illness-script accounts.
- Trajectory observability: The evaluation separates verifiable instruction compliance from broader semantic quality and measures pre-commitment state rather than duplicating the whole prompt or question.Answer Engineering is described as a trajectory editor limited to information and intermediate decisions that become observable.
Experiment
The experiment evaluates instruction duplication across seven instruction-tuned models, 300 medical multiple-choice questions, eight placement conditions, and 16,800 scheduled cells. It measures protocol completion, lexical exposure, answer accuracy, commitment timing, and robustness using factorial and paired analyses.
- Design: 7×300×8 = 16,800 scheduled cells span seven models, 300 medical questions, and eight system/before/after instruction-placement conditions.The frozen run completed 16,646 generations, with 152 truncations and 2 hard failures.
- Protocol: The instruction requests eight ordered headings and postpones answer selection until the provisional-answer section, operationalizing contrastive diagnostic reasoning.The protocol asks models to compare a leading diagnosis with a strong competitor, identify discriminating evidence, and reconsider before committing.
- Measurements: Section completion counts identifiable non-trivial requested sections, while deterministic role completion applies content tests across seven roles.Accuracy is the final multiple-choice answer under intention-to-treat, and premature commitment is evaluated among completed generations.
- Measurements: Pre-provisional TF–IDF recall measures question-stem lexical content recovered in Facts and Implications before provisional-answer selection, rather than factual completeness or semantic understanding.Eligibility is fixed at the question level and reused across models and conditions; the frozen IDF table covers 15,103,887 PubMed abstracts.
- Estimands: The pooled copy-count estimand averages three two-copy conditions minus three one-copy conditions within each model-question block.The analysis also reports all condition means, factorial effects, placement contrasts, and trailing-copy contrasts.
- Contrasts and robustness: Higher-copy cells are tested for departures from additivity using observed-minus-predicted residuals from an additive model fit to zero- and one-copy cells.Question-cluster sign flips provide the residual test, which is diagnostic for super-additivity rather than a linear mechanistic model.
- Contrasts and robustness: Length robustness regressions estimate copy effects while controlling for pre-answer content-token count within question-model fixed effects.Targeted trailing-copy pairs receive question-model-pair fixed effects and question-cluster robust standard errors.
- Multiplicity: 10,000 bootstrap resamples and 50,000 paired sign-flip draws support confidence intervals and p-values, while six pooled outcomes receive joint Holm correction.All-8 completion is reported separately as a derived conjunction, and model-specific diagnostics are corrected across seven models.
Results
Instruction duplication improves several observable protocol measures, especially All-8 completion and TF–IDF recall, while aggregate accuracy remains unchanged. Gains are placement-sensitive and accompanied by increased premature commitment, with the strongest operational evidence appearing in downstream trajectory editing.
- Protocol measures: 73.44% to 74.81%: mean TF–IDF recall rises by 1.38 points with two instruction copies, with diminishing returns across placements.The factorial decomposition indicates saturation rather than additive gains; location and interaction terms are significant for TF–IDF.
- Protocol measures: 90.22% to 93.17%: deterministic All-8 completion rises by 2.95 points, reducing machine-detected failures by 30.2%.TF–IDF recall and other deterministically scored protocol measures also improve after Holm correction.
- Accuracy: 60.21%: aggregate answer accuracy is identical with one and two copies, despite changes in correct-placement counts across model–question blocks.The intervention changes the visible control surface without improving the pooled task endpoint.
- Placement: Trailing duplication raises TF–IDF recall from 73.33% to 75.53% and All-8 by 3.31 points, while accuracy remains unchanged.The effect is positive across all seven models for TF–IDF and All-8 point estimates, rather than being driven by one outlier.
- Audit: The blinded audit finds 10 directional confirmations and 20 perceptual ties, so its prespecified 28/30 confirmation criterion is not met.The results distinguish human-semantic equivalence from controller-operational equivalence, which may depend on explicit alternatives or markers.
Discussion
Instruction duplication changes explicit, machine-addressable trajectory state without changing aggregate answer accuracy, and its operational value depends on the downstream controller and placement. The AE results show gains for SSNHL but a within-AE decrease for conductive branch preservation.
- Control-relevant dissociation: 30.2% of All-8 failures remaining after one instruction are removed without moving aggregate accuracy.The intervention changes the visible control surface rather than the pooled task endpoint.
- Control-relevant dissociation: Copy count and placement independently control the exposed trajectory, with later copies producing smaller, placement-dependent effects consistent with saturation.The factorial analysis identifies location as part of the relevant control parameter.
- Operational interpretation: Formal compliance measures explicit machine-addressable state, not calibrated human-perceived reasoning quality.A deterministic controller cannot inspect or repair omitted qualifiers, alternatives, or reconsideration markers.
- Downstream Answer Engineering: 97.1% was reached for the reproduced SSNHL AE endpoint with a trailing duplicate, versus 84.2% for system-only AE.The published reason-first no-editing endpoint was 25.1%, while trajectory editing raised it to 83.5%.
- Downstream Answer Engineering: 73.8% conductive branch preservation with duplication remained 14.9 points above the published no-editing baseline but below the 78.6% system-only AE reproduction.The authors propose that duplication changed the initial trajectory distribution, leaving fewer locally recognizable conductive trajectories for the existing rules to improve.
- Design implications: Controller-facing evaluation should measure task outcomes, observable downstream-consumed states, and adverse changes together.The study recommends testing duplication against the controller’s own endpoint and retaining an adverse-effect check.
Limitations and Ethics
The study’s evidence is bounded by selected medical benchmarks, near-saturated protocol metrics, exploratory measurement choices, and a nonconfirmatory human audit. Its benchmarks contain no identifiable patient data, but the results do not establish clinical safety or medical competence.
- Scope: Transfer beyond seven selected contemporary models and three medical multiple-choice datasets remains unestablished.The authors recommend testing other domains, instruction families, serving stacks, and semantically equivalent paraphrases.
- Measurement boundaries: Several protocol metrics are near saturation after one instruction, while All-8 retains more headroom because any marginal role failure causes conjunction failure.The human audit indicates that many threshold crossings are perceptually small.
- Measurement boundaries: TF–IDF measures lexical exposure rather than understanding, and All-8’s relative failure reduction does not enlarge its +2.95-point absolute effect.Length-adjusted regressions are descriptive robustness checks because they condition on a post-treatment variable.
- Validation: The blinded challenge audit used one non-clinician author, oversampled machine-positive changes, and failed its prespecified 28/30 confirmation criterion.The AE evidence is a downstream supporting experiment rather than a mediation test.
- Ethics: The public benchmarks contain no identifiable patient data, and no external human participants were recruited.These results are not evidence of clinical safety or medical competence.
- Measurement and analysis: Final analyses use a simpler observable criterion after a stricter semantic counterfactual classifier was dropped, and the analyses are exploratory rather than preregistered confirmation.Models were frozen before post-generation refinement of deterministic measurement.
- Reproducibility: Frozen-output verification and deterministic rejudging audit the reported run offline, whereas model regeneration tests replication under a potentially changed serving stack.The AE artifact preserves paired cases and scoring/runtime artifacts for auditability.
Conclusion
Instruction duplication is a placement-sensitive inference-time control that improves machine-addressable trajectory state while leaving aggregate answer accuracy unchanged and increasing premature commitment. Its practical value emerges when a downstream controller consumes the trajectory, but effects can differ across branches.
- Core findings: 90.22% to 93.17%: deterministic All-8 rises with two copies, while aggregate answer accuracy remains 60.21%.Pre-commitment TF–IDF exposure rises from 73.44% to 74.81%, and premature commitment rises from 1.52% to 2.30%.
- Downstream consequence: 97.1% SSNHL performance with duplication exceeds the 84.2% reproduced system-only AE result, while 73.8% conductive preservation is below 78.6% system-only AE.The conductive duplicated system nevertheless remains above the 58.9% published no-editing baseline.