PaperlayerView exact revision on arXiv

Source-linked AI summary

LangSelect: Cost-Aware Target-Language Routing for LLM Code Generation

Son Ha Xuan, Phat T. Tran-Truong, Xuan-Bach Le, Nghia Duong-Trung

arXiv:2609.18959v1cs.CL

TL;DR

Code-generation systems usually fix the output language before decoding, although language-flexible tasks can have accepted implementations with different token costs. LangSelect routes among languages before generation and verifies attempts with fallback; live evaluation finds a cost-and-pass trade-off between its Domain heuristic and learned selector.

  • Problem

    Code-generation systems commonly fix the requested output language before decoding, despite language-flexible tasks allowing several languages to pass the same tests and differ in token cost.

    Source: §1 · p. 1

  • Method

    LangSelect ranks languages from task features, verifies generated code, falls back after failures, and is evaluated both by replaying accepted solutions and by live generation charging every attempt.

    Source: §5 · p. 5

  • Results

    50.3% harness-proxy savings at 92.9% pass after fallback for the Domain heuristic; CodeBERT+metadata reaches 93.8% pass after fallback with a 3.7% harness-proxy token increase.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Output language can be a routing decision for function-level, unit-test-verifiable workloads, with the allowed languages, verifier, fallback budget, and accounting rule defining the routing boundary.

    Source: §9 · p. 8

    The reported savings concern function-level, unit-test-verifiable code generation; repository-pinned edits, file-spanning refactors, and language-mixed pipelines require separate evaluation.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM code-generation systems usually choose a target programming language before decoding and treat that choice as fixed. We show that, for language-flexible programming tasks -- tasks where several target languages are acceptable and checkable by the same tests -- this choice is a measurable cost lever: verified implementations of the same task can differ substantially in generated-token length. We introduce LangSelect, a verification-aware router that selects the target language before generation and falls back when the first attempt fails. To separate offline routing opportunity from end-to-end behavior, we evaluate verified-solution replay, which chooses among already accepted corpus solutions, and live GPT-5 generation, which charges every generation attempt, including failures and fallbacks. On MultiLang-Bench, a 3,000-task, 8-language verified corpus, replay shows substantial language-routing headroom. In live evaluation on 450 held-out tasks, a train-split Domain heuristic baseline reduces harness-proxy tokens, which include wrapper and entrypoint overhead, by 50.3\% at 92.9\% pass after fallback, while a learned CodeBERT+metadata selector reaches the highest pass after fallback, 93.8\%, with a 3.7\% token increase. These results show that output-language routing can define a practical cost-correctness frontier for unit-test-verifiable code generation.

1 Introduction

For language-flexible tasks, target-language choice is a measurable token-cost lever because accepted implementations vary in length. LangSelect ranks languages before generation, verifies attempts, and falls back when needed.

1 Introduction

  • In iterative code-generation systems, each failed attempt adds generated-code tokens, so cost depends on the retry trace as well as the final program.Prior efficiency work commonly fixes the requested output language before decoding.Source: §1 · p. 1
  • 2.19× is the average difference between the cheapest and most expensive accepted implementations across 3,000 tasks and 8 languages.The group-by example ranges from 12 tokens in Clojure to 60 in Rust.Source: §1 · p. 1
  • LangSelect ranks target languages from task features, generates in the top-ranked language, verifies the result, and retries after failure.The evaluation separates replay among already accepted solutions from live generation, which charges every attempt, including failures and fallbacks.Source: §1 · pp. 1–2
  • 50.3% harness-proxy savings at 92.9% pass after fallback are achieved by the Domain heuristic, while CodeBERT+metadata reaches 93.8% pass with a 3.7% token increase.Both policies add under 1% latency; the Domain heuristic is close to the non-deployable Hindsight oracle (+49.6% / 92.7%).Source: §1 · p. 2
  • MultiLang-Bench contributes a verified corpus of 3,000 tasks across 8 languages, while LangSelect supports heuristic, supervised, and bandit policies.The paper formulates routing under an explicit token-cost and pass-rate boundary.Source: §1 · p. 2
  • Language routing complements model cascades, which vary the model, and multilingual benchmarks, which assess whether models solve tasks across languages.LangSelect asks which acceptable language to attempt first [Chen et al., 2021; Austin et al., 2021].Source: §1 · p. 2

2 Motivating Evidence

Verified function bodies show substantial average token variation across the corpus, but the amount of routing headroom depends on task family.

2 Motivating Evidence

  • 2.19× is the average spread between the cheapest and most expensive unit-test-verified function bodies across 3,000 tasks and 8 languages.Character- and byte-level audits show similar variation, so the effect is not merely an artifact of tokenization.Source: §2 · p. 2
  • 1.33×–1.50× are the row-wise median spreads across six task families, and the cheapest language varies by family, often between Go and Python.The pattern supports task-dependent rather than globally fixed language choice.Source: §2 · p. 3
  • Math, string, and I/O-converted tasks have the largest measured cheapest-language headroom, while numeric tasks have the smallest.Numeric-heavy deployments should therefore expect smaller gains than workloads dominated by string processing or I/O-style tasks.Source: §2 · p. 3

3 Problem Formulation

The formulation makes routing comparisons explicit by fixing the generator, language portfolio, verifier, and token-count rule, then constraining cost against a Python pass-rate reference.

3 Problem Formulation

  • A routing comparison fixes the generator, allowed languages, verifier, and token-count rule, and evaluates cost and strict pass@1 for each task-language choice.The policy seeks lower expected cost subject to a Python-relative pass constraint.Source: §3 · p. 3
  • Replay has zero pass-rate shortfall by construction, while live targeted generation measures the shortfall alongside cost.The hindsight oracle selects the cheapest passing language within the verified corpus and is a within-corpus reference, not a universal upper bound.Source: §3 · p. 3
  • The selector receives attempt-level feedback: passing attempts using fewer tokens earn more credit, while failures earn zero.When fallback succeeds, only the passing language receives credit, although earlier failed attempts still count toward task token cost.Source: §3 · p. 3

4 LangSelect Framework

LangSelect scores available languages from pre-generation task information, verifies generated code, and uses verification outcomes to guide fallback and policy updates.

4 LangSelect Framework

  • The routing loop encodes each task, scores languages, generates with a language-conditioned prompt, verifies with tests and language-fidelity checks, and records outcomes.Execution supplies an empirical signal of whether a routing choice succeeded.Source: §4 · p. 3, Figure 2 · p. 4
  • Deployable metadata sets a coarse language prior, then CodeBERT embeddings and structured metadata refine it, while Neural-LinUCB heads add uncertainty-aware selection.Metadata includes domain, source group, argument/return type family, and difficulty.Source: §4 · pp. 3–4
  • After a failed first attempt, LangSelect tries the next-ranked language for up to three total attempts, with Python reserved as the final fallback.Token-spread features from cross-language counts are used for auditing, not deployment.Source: §4 · p. 4
  • Deployment constraints can restrict the policy with hard portfolio masks or apply soft penalties for maintainability and task-specific integration costs.The quality term can favor predicted pass probability, and deployment-time weights control the penalties.Source: §4 · p. 4

5 Experimental Setup

The experiments use a harness-verified, multilingual corpus and separate replay over accepted solutions from live generation, where every attempt is charged. Comparisons hold generation and verification settings fixed while varying the routing policy and token-accounting boundary.

Corpus

  • MultiLang-Bench contains 3,000 deterministic pure-function tasks, each with an accepted solution in eight languages; all 24,000 solutions pass unit-test and language-fidelity checks.The fixed split is 2,100/450/450 for train, validation, and test.Source: §5 · p. 5

Evaluation protocols

  • Replay selects among already verified solutions, so strict pass@1 is 100% by construction; live targeted generation tests fresh GPT-5 outputs and charges failed and fallback attempts.Replay isolates routing and accounting from generation noise, while live evaluation measures deployable recovery after generation.Source: §5 · p. 5

Accounting

  • All reported generations use GPT-5, with token counts evaluated at function-body, function-plus-signature, and harness-proxy boundaries.Harness-proxy accounting includes wrapper and entrypoint overhead.Source: §5 · p. 5

Baselines

  • Deployable baselines use only pre-generation task fields and training/validation data; the Hindsight oracle uses test-task verified costs and is a non-deployable reference.Live comparisons share prompts, GPT-5 settings, verifier, token counter, and fallback budget; only routing policy changes.Source: §5 · p. 5

Verification scope

  • “Verified” means accepted by the unit-test and language-fidelity harness, so replay covers accepted corpus solutions rather than all possible programs.Accepted solutions may come from translation, curation, or author review, but each must pass the full harness.Source: §5 · p. 5

6 Results

Replay measures cost opportunity among accepted solutions, while live generation charges fresh GPT-5 attempts, including failures and fallbacks. The results span replay headroom, supervised selectors, live cost–correctness trade-offs, operational sensitivity, and constrained language portfolios.

6.1 Replay headroom

  • 42.1% function-body reduction is the MultiLang-Bench Hindsight oracle result; Hybrid prior+LinUCB reduces cost by 28.3% on HumanEval-X, 31.6% on MultiPL-E, and 34.2% on McEval.The Hybrid results are under function-body accounting and each has p < 0.001 after Bonferroni correction.Source: §6.1 · p. 6

6.2 Supervised baselines

  • +5.83% cost reduction and 61.5% oracle recovery are the random-forest supervised baseline results; replay charges a wrong first language as extra cost rather than retrying.Oracle recovery is the fraction of tasks whose first selected language matches the hindsight-optimal language.Source: §6.2 · p. 6

6.3 Live targeted generation

  • +50.3% harness-proxy cost reduction at 92.9% pass after fallback for Domain heuristic; CodeBERT+metadata reaches 93.8% pass after fallback with a 3.7% harness-proxy token increase.The Domain heuristic is close to the non-deployable Hindsight oracle (+49.6% / 92.7%); Figure 4 shows these distinct operating points.Source: §6.3 · p. 6, Figure 4 · p. 7

6.4 Sensitivity, latency, and cold start

  • 39.65 ms mean and 50.63 ms p95 routing overhead remain below 1.01% of GPT-5 generation latency; Neural-LinUCB regret stabilizes after roughly 50 tasks.Across five cost views and seven reward normalizations, the selected language is unchanged per task; the accounting boundary changes savings magnitude but not direction.Source: §6.4 · p. 7

6.5 Deployment Constraints

  • 15.15% harness-proxy replay headroom remains for the mainstream Python, Java, Go, and Ruby portfolio; removing C and Rust costs only 0.1 pp.The full eight-language portfolio gives 19.09% harness-proxy headroom.Source: §6.5 · p. 7, Table 3 · p. 8

7 Analysis

On this task distribution, coarse domain structure captures most live cost savings, while learned selection and fallback offer a different pass-rate operating point.

Analysis

  • Domain heuristic captures most live cost saving, while CodeBERT+metadata reaches the highest pass rate and gains most from fallback; their distinct trade-offs reflect pass-rate risk and fallback-budget choices.The cost-optimal language varies by task type, with Ruby for string-heavy tasks, Clojure for math, C for systems-style code, and Go or Java for structured tasks.Source: §7 · p. 8

8 Related Work

LangSelect complements prompt-, model-, and decoding-side efficiency methods by changing the requested implementation language while holding the task and generator fixed. It frames language choice as a contextual, verifier-checked routing decision under a generated-token budget.

Complementary efficiency methods

  • LangSelect complements methods that change prompts, models, or decoding by holding the task and generator fixed while changing the requested implementation language.The related work includes prompt compression, terse reasoning, model routing, and triage-style coding-agent routing.Source: §8 · p. 8

Benchmark context

  • The work asks which verified language to attempt first under a generated-token budget, building on unit-test and multilingual code-generation benchmarks.The cited benchmark progression ranges from HumanEval and MBPP to APPS, BigCodeBench, DS-1000, HumanEval-X, MultiPL-E, and McEval.Source: §8 · p. 8

Contextual routing

  • A globally shortest-language rule is insufficient because tokenization, programming-language confusion, and idiomatic concision vary; LangSelect instead treats language choice as contextual and checked by verification.Its LinUCB variant builds on contextual-bandit work.Source: §8 · p. 8

9 Conclusion

Output programming language can be a routing decision for function-level, unit-test-verifiable workloads. Live evaluation shows a cost–pass frontier, while replay and live generation measure different aspects of routing.

9 Conclusion

  • 50.3% harness-proxy savings at 92.9% pass after fallback are achieved by the Domain heuristic, while CodeBERT+metadata offers a higher-pass point on the cost–pass Pareto frontier.The Domain heuristic is within 1 percentage point of the non-deployable Hindsight oracle.Source: §9 · p. 8
  • The paper frames output-language routing as a practical cost–correctness frontier for function-level workloads that can be checked by unit tests.The authors do not propose replacing Python.Source: §9 · p. 8
  • Replay isolates routing over verified solutions, whereas live generation charges failed and fallback attempts, so a hindsight oracle does not stand in for a deployable policy.The practical routing boundary depends on the allowed languages, verifier, fallback budget, and accounting rule.Source: §9 · p. 8

Limitations

The reported evidence is bounded by function-level tasks, GPT-5 live generation, and the stated cost-accounting and verification choices. Several deployment settings and evaluation extensions remain open.

Limitations

  • The four benchmarks target function-level, unit-test-verifiable generation; repository-pinned edits, file-spanning refactors, and language-mixed pipelines require separate evaluation.MultiLang-Bench focuses on polyglot-solvable pure functions to enable controlled per-task replay.Source: Limitations · p. 9
  • Live results use GPT-5, and each additional model requires its own traces despite the framework being model-agnostic.Evaluation with Claude, Gemini, and open-weight models is identified as follow-up work.Source: Limitations · p. 9
  • A mainstream-only portfolio retains 15% replay headroom, but practitioners must choose a portfolio matching their maintainability and policy constraints.The portfolio restriction covers Python, Java, Go, and Ruby.Source: Limitations · p. 9
  • Reported savings use three accounting boundaries and exclude imports, build metadata, and downstream integration code; tests and language-fidelity checks are not semantic proofs.The most conservative boundary is headlined.Source: Limitations · p. 9
  • Multimodel live tests, mainstream-only live portfolios, cold-start live ablations, and repository-level evaluation are prioritized open directions.Further directions include full-file accounting and latency-aware fallback.Source: Limitations · p. 9

A Supplementary Experimental Protocol

The supplementary protocol defines the datasets, comparison policies, attempt accounting, and execution controls behind the reported results. It also documents contamination checks and the limits of the language-fidelity detector.

A Supplementary Experimental Protocol

  • The protocol audit uses 3,000 verified tasks split into 2,100 training, 450 validation, and 450 test tasks, with one accepted solution per task in eight languages.The 24,000 retained solution records pass the local harness, expected-entrypoint check, and language-fidelity check.Source: A Supplementary Experimental Protocol · p. 11
  • Replay selects among already-passing corpus solutions, while live targeted generation charges every fresh-generation attempt, including failures and fallbacks.Replay has strict pass@1 of 100% by construction; the Hindsight oracle is a within-corpus reference based on known test-task solution costs.Source: A Supplementary Experimental Protocol · p. 11
  • Domain heuristic has 87.3% first-pass success and triggers fallback on 12.4% of tasks, while CodeBERT+metadata has 91.8% first-pass success and triggers fallback on 8.0%.Python-only has 92.0% first-pass success and never falls back; Hindsight oracle has 87.1% first-pass success and triggers fallback on 12.7%.Source: A Supplementary Experimental Protocol · p. 11
  • Deployable policies use only train/validation data and pre-generation task fields, with shared prompts, GPT-5 settings, verifier, token counter, and fallback budget.Python-only is fixed; Source and Domain heuristics rank languages by aggregate verified token cost within source group or task domain.Source: A Supplementary Experimental Protocol · p. 11
  • Learned selectors use metadata, frozen CodeBERT task-specification embeddings, or their concatenation to rank first-choice and fallback languages without test-time cost labels.Metadata includes source group, domain, difficulty, and argument/return family.Source: A Supplementary Experimental Protocol · p. 11
  • Table 4 audits contamination between the MultiLang-Bench training split and external multilingual benchmark families, and the main-paper claims do not depend on dataset overlap.Exact, near-duplicate, and semantic overlaps are removed before evaluation.Source: Table 4 · p. 12, A Supplementary Experimental Protocol · p. 12

A.1 Execution, Decoding, and Language-Fidelity Controls

  • Recovery decoding permits up to three attempts at temperature 0.8, charges failures and fallbacks, and runs generated code in fixed, isolated language-specific harnesses.Compilation errors, runtime exceptions, timeouts, and failing unit tests count as failed attempts.Source: §A.1 · p. 12
  • The audit reports live traces on 450 held-out GPT-5 tasks under three accounting boundaries and cost-model and reward-normalization checks.The language-fidelity detector combines parse/compile checks, wrappers, entrypoint validation, and executor compatibility, but subtle mixed-language fragments may remain.Source: §A.1 · p. 12

B Additional Formulation and Method Details

This appendix documents LangSelect’s formulation, methods, benchmark and cost-measurement boundaries, and supplementary analyses. It also qualifies the routing results through accounting and latency analyses and describes complementary routing approaches, diagnostics, and reproducibility limits.

B.1 Language Selector: Neural-LinUCB Architecture

  • Neural-LinUCB shares task features across languages and adapts per-language linear heads; live GPT-5 results put Domain heuristic at +50.3% saving and 92.9% pass, versus CodeBERT+metadata’s 93.8% pass.The Domain heuristic is within 1 percentage point of the Hindsight oracle’s +49.6% saving and 92.7% pass; these operating points span the cost–correctness frontier.Source: §B.1 · p. 13

B.2 Warm-Start Procedure; D.1 Supervised routing baselines

  • Warm-start improves early language-choice reward and shortens convergence to roughly 50 tasks; six supervised baselines use a shared CodeBERT-plus-metadata feature vector.The diagnostic convergence result is from n=204; the supervised-baseline audit reports n=39 held-out warm-start examples.Source: §B.2 · pp. 13–14, §D.1 · p. 17

B.3 Verification and Fallback Loop

  • Verification retries failures, including off-target generations, for up to three total attempts, reserving Python as a final fallback when untried and charging every attempt.Only a passing attempt is returned; if all attempts fail, the task is reported as failed, and each attempted language receives its own bandit observation.Source: §B.3 · p. 14

B.4 Reward Normalization

  • Reward maps token counts to [0, 1] using warm-start-corpus 5th and 95th percentiles fixed per tokenizer–dataset pair; only passing attempts receive positive, cost-decreasing reward.Failed or off-target outputs receive zero reward.Source: §B.4 · p. 14

B.5 Idealized Theoretical Statements

  • The regret statement concerns idealized calibrated LinUCB assumptions, not a finite-sample guarantee for implemented Neural-LinUCB; verifier-conditional correctness covers evaluated tests only when an attempt passes.When pass probabilities are equal, maximizing expected pass-gated reward is equivalent to minimizing expected normalized cost; when they differ, regret is reward regret rather than raw-token regret.Source: §B.5 · pp. 14–15

B.6 Token Boundary Sensitivity; C MultiLang-Bench Construction Details

  • The main token metric counts generated function bodies, excluding wrapper and harness code; MultiLang-Bench contains 3,000 verified tasks with accepted solutions in eight languages.The function-body boundary improves cross-language comparability, but full-file savings may be smaller for compiled or package-heavy languages.Source: §B.6 · p. 15, C MultiLang-Bench Construction Details · p. 16

D.2 Replay ablation and oracle recovery

  • Replay achieves 100% strict pass@1 by construction; the hindsight oracle shows 19.09% harness-proxy and 42.1% function-body headroom on the warm-start corpus.The replay ablation uses n=377 under function-body accounting; live targeted-generation pass rates are reported separately.Source: §D.2 · p. 17

D.3 Cost-model and reward-normalization sensitivity

  • Hindsight saving remains 18.94% across five cost views, and the seven reward normalizations preserve selected languages and reported reductions.The sensitivity tables rescale the same verified outputs or keep the selected languages unchanged.Source: §D.3 · p. 17, Table 7 · p. 18, Table 8 · p. 18

D.4 Warm-start label sensitivity

  • Under harness-proxy accounting, Ruby is the warm-start oracle for 112/377 tasks (29.7%) and Python for 170/377 (45.1%); Ruby’s average saving against Python falls from 45.2% to 18.3%.Under function-body accounting, Ruby is the oracle for 353/377 tasks (93.6%); the paper uses harness-proxy accounting as its headline measure because it is closer to deployed inference cost.Source: Table 9 · p. 18, §D.4 · p. 18

D.5 Domain-level headroom partitions

  • Table 10 partitions verified-corpus hindsight headroom by held-out test set, while Table 11 audits per-domain function-body token costs for DH, CB+meta, and hindsight, where lower is better.The dominant-language column is a verified-corpus property, Table 10 shows only the lowest and highest token-spread quartiles, and Table 11 sums costs over tasks in each domain (n=373).Source: §D.5 · pp. 18–19, Table 10 · p. 19, Table 11 · p. 19

D.6 Selector latency decomposition

  • The selector’s CodeBERT embedding dominates overhead, but total selector time remains below 1% of GPT-5 generation latency.The full timing log reports metadata extraction at 0.015/0.042 ms, CodeBERT embedding at 39.217/56.001 ms, and LinUCB scoring at 0.280/0.401 ms (p50/p99).Source: Table 12 · p. 19, §D.6 · p. 19

D.7 Composability with model routing

  • Language routing and model routing act on different variables, and their savings can compose multiplicatively when measured under a compatible accounting boundary.Using harness-proxy rℓ = 0.503 and illustrative rm = 0.4 yields an estimated combined saving of ≈0.702; the paper did not run a joint experiment.Source: §D.7 · p. 19

E Related Work Taxonomy

  • LangSelect holds the model and prompt family fixed while routing the output language; it reports +50.3% live harness-proxy savings and 42.1% verified replay headroom on language-flexible tasks.Language choice is not reducible to a static shortest-language rule: drift, tokenization, and semantic differences affect concision, motivating contextual-bandit selection with verification and fallback (Moumoula et al., 2025; Sennrich et al., 2016; Kudo and Richardson, 2018; Petrov et al., 2023; Nanz and Furia, 2015; Bostrom and Durrett, 2020; Li et al., 2010; Zhou et al., 2020).Source: E Related Work Taxonomy · p. 20

F Extended Results

  • The 204-task adaptive trace shows Neural-LinUCB accumulating less regret than adaptive LinUCB, ε-greedy, and random diagnostics, but it is not a headline result.The paper identifies held-out replay and live targeted generation as the basis for its main empirical claims.Source: F Extended Results · p. 20, Figure 6 · p. 20

G Reproducibility Checklist

  • The reproducibility checklist fixes the dataset, language portfolio, model and token counter, selector features, seeds, fallback budget, and execution boundary.It specifies 3,000 tasks across 8 languages with a 2,100/450/450 split; replay passes by construction, while live attempts charge failures and fallback retries; accepted solutions run in fixed local harnesses with network access disabled.Source: G Reproducibility Checklist · p. 21

Related papers

Loading 2609.18959v1…