IntentCoding: Amplifying User Intent in Code Generation

• Standard decoding can satisfy some specified constraints yet fail others across a broader range of inputs.The CodeLlama example satisfies return-type and element-type requirements but misses the lower bound and required list length.Source: Figure 1 · p. 1, §1 · p. 2
• Performance deteriorates as user-specified constraints increase, while their influence on logits can be too weak to guide standard decoding effectively.Source: §1 · p. 2
• IntentCoding contrasts logits from the original and intent-masked prompts, then amplifies intent influence using multiple scales ensembled during beam search.It is model-agnostic and training-free.Source: §1 · p. 2
• CodeConstraints combines primitive constraints and varies their number to evaluate intent compliance under differing task difficulty.Its construction is described as extensible and adaptable to new constraint types and dataset sizes.Source: §1 · p. 2
• Experiments on HumanEval, LiveCodeBench, IFEvalCode, and CodeConstraints consistently demonstrate IntentCoding’s effectiveness and superiority.Source: §1 · p. 2

2.1 Code Generation with LLMs

• Execution-guided and refinement methods use tests, compilers, or runtime feedback, which can limit applicability where those signals are unavailable.Source: §2.1 · pp. 2–3
• Benchmarks span standalone function synthesis, broader programming problems, and recent contest tasks, but mainly assess functional correctness rather than multi-constraint satisfaction.Source: §2.1 · p. 3

2.2 Decoding Methods

• Decoding approaches include greedy and beam search, sampling variants, adaptive temperature, and Selective Prompt Anchoring.Source: §2.2 · p. 3
• Unlike SPA, IntentCoding avoids searching for a single best strength, while fixed-strength performance can be sensitive to tuning.SPA depends on a calibration split; fixed α performance varies across models and α values.Source: §2.2 · p. 3

3.1 Construction of the CodeConstraints

• CodeConstraints addresses benchmarks’ inability to assess simultaneous constraints by varying four primitive constraint types across difficulty levels, with accuracy requiring every constraint to be satisfied.The primitives are data type, return format, length, and value.Source: §3.1 · pp. 3–4

3.2 Empirical Observations

• 93% to 23%: DeepSeek-Coder’s accuracy on CodeConstraints falls from two to four constraints, with similar declines for other LLMs.Figure 2 shows 7B model performance across complexity levels using greedy search.Source: Figure 2 · p. 4, §3.2 · p. 4
• Even when user intent shifts token logits toward the correct choice, the incorrect token can remain selected under greedy decoding.In the analyzed example, the correct token’s logit rises from 0.242 to 0.359, while the incorrect token’s logit falls from 0.419 to 0.407 but remains higher.Source: §3.2 · p. 4

3.3 The Proposed Intent↑Coding Method

• Intent↑Coding contrasts original and intent-masked logits to isolate the intent signal, scales it across multiple α values, ensembles candidate tokens, and expands beam hypotheses.The method integrates the token-level ensemble into beam search, retaining hypotheses with the highest cumulative log-probabilities.Source: §3.3 · pp. 5–6

4.1 Experimental Setup

• Constraint following is evaluated on CodeConstraints and IFEvalCode, while general code generation is evaluated on HumanEval and LiveCodeBench using three open-source LLMs.The models are CodeLlama, DeepSeek-Coder, and Qwen2.5-Coder.Source: §4.1 · p. 6

4.2 Main Results

• IntentCoding outperforms baseline decoding across LLMs and benchmarks, with average relative improvements of 16.9% to 34.6%; constraint benchmarks show 30.5% to 71.0% gains, alongside gains in functional correctness.The general code-generation benchmarks include HumanEval and LiveCodeBench.Source: §4.2 · pp. 6–7

4.3 Comparing Different Beam Sizes

• IntentCoding with beam size one reaches 63% accuracy on Qwen2.5-Coder, versus 46% for greedy search, and maintains high, stable performance at larger beam sizes.Standard beam search yields only modest gains and may degrade at larger settings.Source: §4.3 · p. 7, Figure 5 · p. 7

4.4 Comparing Our Approach with a Fixed α

• IntentCoding’s predefined token-level ensemble consistently outperforms the best fixed α for each LLM without manual tuning.The best fixed α differs across models, and fixed-α performance can vary substantially within a model.Source: §4.4 · p. 7, Figure 6 · p. 7

4.5 LLMs of Different Mask Settings

• Masking an individual constraint consistently improves performance on that targeted constraint across LLMs and constraint types.The experiments include masking variable values and collection length.Source: §4.5 · pp. 7–8, Figure 7 · p. 8

4.6 Performance with Different LLMs

• IntentCoding improves instruction-tuned models and consistently outperforms standard decoding across model sizes from 1.5B to 33B.Accuracy rises from 31% to 53% for CodeLlama-IT and from 52% to 69% for DeepSeek-Code-IT; Qwen2.5-Coder-1.5B improves from 11% to 17%.Source: §4.6 · p. 8, Table 2 · p. 8, Table 3 · p. 8

5 Conclusion

• IntentCoding is a decoding strategy designed to improve LLMs’ ability to follow user intent in multi-constraint code generation.It extracts and amplifies the influence of user intent through a multi-strength ensemble without additional training or model modification.Source: §5 · p. 8
• CodeConstraints is a benchmark for testing LLMs under varying levels of constraint complexity.Source: §5 · p. 8
• Experiments across multiple LLMs and datasets report improved functional correctness and intent compliance.Source: §5 · p. 8

Limitations

• CodeConstraints covers four relatively simple constraint types, limiting its coverage of more complex real-world requirements.The authors identify stylistic, architectural, and repository-specific constraints as possible additions.Source: Limitations · p. 9
• IntentCoding requires batching the original prompt with an intent-masked variant, introducing compute overhead that the authors describe as modest in practice.Source: Limitations · p. 9
• Direct deployment through popular closed-source APIs is difficult when they do not expose token-level logits or provide only limited information.The experiments therefore primarily focus on open-source LLMs.Source: Limitations · p. 9

A.1 Datasets

• HumanEval evaluates code-generation capabilities using 164 handcrafted Python problems, while LiveCodeBench uses programming-contest problems evaluated against test cases.The experiments use LiveCodeBench Version 5, containing 279 easy, 331 medium, and 270 hard problems.Source: §A.1 · p. 11
• IFEvalCode pairs code-generation tasks with explicit, verifiable constraints and an executable checker, and the experiments report its Instr. metric on Python and English questions.Source: §A.1 · p. 11
• CodeConstraints evaluates compliance with user-specified constraints, and the experiments use Level 4, with four constraints per sample.Source: §A.1 · p. 11

A.2 Base Models

• The experiments use CodeLlama-7B-hf, a variant from the code-specialized CodeLlama family.The family is based on LLaMa and further pre-trained on permissively licensed code.Source: §A.2 · p. 11
• The experiments use DeepSeek-Coder-6.7B-Base, from a family trained from scratch on 2 trillion tokens, 87% code and 13% natural language.Source: §A.2 · p. 11
• The experiments use Qwen2.5-Coder-7B, a code-specific model series extending Qwen2.5 with specialized training on 5.5 trillion tokens.Source: §A.2 · p. 11

A.3 Implementation Details

• For standard beam search and IntentCoding, beam size is 4; sampling methods use specified top-p and temperature settings, with sampling results averaged over three runs.AdapT uses top-p 0.95 and a grid search over T1 and T2; Select Prompt Anchoring reports its best anchoring-strength configuration.Source: §A.3 · p. 11
• Maximum generation length is 512 for HumanEval and IFEvalCode, 1024 for LiveCodeBench, and 256 for CodeConstraints.The reported metrics are pass@1 for HumanEval and LiveCodeBench, and accuracy requiring all constraints to be satisfied for CodeConstraints.Source: §A.3 · p. 11

A.4 Performance with Different LLMs Settings

• The baseline is the better greedy- or beam-search performance, with beam sizes 1–4 searched; subscripts denote IntentCoding’s relative improvement over that baseline.Source: §A.4 · p. 11

B Inference Efficiency

• IntentCoding is slightly slower per token than greedy and beam search but often generates fewer tokens than beam search, making total inference time lower than beam search.The authors attribute the per-token slowdown to ensembling multiple contrastive strengths.Source: B Inference Efficiency · p. 12

C Case Studies

• In a four-constraint example, IntentCoding satisfies all constraints and terminates at the intended logic, unlike greedy decoding and beam search.Greedy misses the correct value and set size; beam search fixes the value but not the exact size and adds unrelated code.Source: C Case Studies · p. 12

D Token-Level Ensemble Details

• At each decoding step, IntentCoding takes the top-1 token from each of six logit distributions and groups candidates that share a token ID.It assigns each unique token the average probability of the candidates in its group.Source: D Token-Level Ensemble Details · p. 12

E Prompt

• IntentCoding automatically uses Python regular expressions to locate and mask the natural-language intent span while preserving the rest of each dataset prompt.The prompt-masking pipeline is fully automated.Source: E Prompt · p. 12

E.1 HumanEval Prompt

• The HumanEval masked prompt places a mask before the task description while retaining the function signature and docstring examples.The provided prompt examples show the original and masked versions of the same task.Source: §E.1 · p. 12

E.2 LiveCodeBench Prompt

• The LiveCodeBench prompt asks for the index of the second-largest element in an integer sequence, with distinct elements and stated input bounds.The prompt specifies 2 ≤ N ≤ 100 and integer inputs.Source: §E.2 · p. 13

E.3 CodeConstraints Prompt

• The CodeConstraints example asks for a tuple of two floats, each less than a given positive integer.The masked prompt is also shown.Source: §E.3 · p. 13

E.4 IFEvalCode Prompt

• The IFEvalCode example combines a character-escaping task with five code constraints, including snake_case names, a list comprehension, and a line limit.The constraints also require a ternary operator and at least one lambda expression.Source: §E.4 · p. 14