Source-linked AI summary
Translating the Translator: Decomposing the Cost of English-Forced Inter-Agent Communication
Kushagra Agrawal, Yuming Feng, Man-Fai Leung
TL;DR
Agent frameworks often route non-English inter-agent communication through English, but whether explicit translation adds loss beyond native routing had not been tested. The paper compares controlled multi-agent pipelines and finds a significant Exact Match penalty for English-forced handoffs, with forward-translation degradation associated with failure.
Problem
Whether forcing multi-agent handoffs through English adds cross-lingual loss compared with native routing, and which stage drives it, remained untested.
Source: §1 · p. 1Method
The study compares three pipeline conditions, including native-language routing and an English-forced pipeline with back-translation, to isolate the end-to-end routing cost.
Source: §1 · p. 2, §3.2 · p. 4Results
Takeaways & Limitations
The results support native-language routing for typologically distant languages, where the English-forcing penalty is substantial.
Source: §7 · p. 8, §8 · p. 8The use of an 8-bit quantized model may affect the penalty’s magnitude, but its direction cannot be determined without an unquantized control.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Multi-agent LLM architectures, such as LangChain and AutoGen, largely assume English as the lingua franca for internal inter-agent communication, even when the end-user task is non-English. We fill this gap by evaluating a two-agent extraction-answer core, with an additional back-translation agent in the English-forced condition, across four typologically diverse languages (Hindi, Chinese, Spanish, Arabic; n = 300 per language) using the Aya-23-8B model. We compare a native-language pipeline to an English-forced one (which incorporates a final back-translation step from English to the user's language). We discover a statistically significant English-Forcing Tax (surviving a strict Bonferroni correction) that isolates the cost of English routing from general multi-agent orchestration overhead. Forcing inter-agent communication through English reduces Exact Match accuracy by 13.0 percentage points (Spanish) up to 30.6 percentage points (Hindi) compared to native-language multi-agent execution. Using chrF scores as a diagnostic measure of English-reference lexical overlap, we find that lower overlap is strongly associated with pipeline failure, consistent with translation loss being an important contributor to the observed performance drop. These findings suggest a compelling case for native-language routing in agent frameworks when the source and target languages are typologically distant, reducing a compounding translation tax.
1 Introduction
The paper tests whether explicitly routing non-English agent handoffs through English adds translation loss beyond ordinary multi-agent overhead. In a controlled comparison, English-forced routing lowers Exact Match accuracy relative to native-language routing, while forward-translation degradation is strongly associated with pipeline error.
1 Introduction
- Agent frameworks commonly hardcode prompts, tool schemas, and natural-language handoffs in English even for non-English user queries.The stated engineering rationale is that models are mostly trained on English data and may reason most strongly in English.Source: §1 · p. 1
- Whether explicit English routing compounds cross-lingual loss relative to native routing, and which pipeline stage primarily drives any loss, remained untested.The question joins known cross-lingual translation loss with information loss in multi-agent communication.Source: §1 · pp. 1–2
- The study evaluates a two-agent extraction-answer core in native and English-forced settings, with an added back-translation agent in the English-forced pipeline.It also includes a single-agent baseline to separate English-routing costs from underlying model weakness and orchestration overhead.Source: §1 · p. 2
- The paper presents a three-condition design to isolate the architectural penalty of English-forced handoffs from the model’s baseline cross-lingual weakness.Its contribution also includes a chrF diagnostic analysis of English-reference lexical overlap and pipeline failures.Source: §1 · p. 2
- Up to 30.6 percentage points of Exact Match accuracy are lost with English-forced versus native-language multi-agent handoffs, and the penalty increases with typological distance from English.The English-Forcing Tax is isolated from baseline multi-agent orchestration overhead; lower English-reference lexical overlap is strongly associated with failure and forward-translation degradation.Source: §1 · p. 2
2 Related Work
Prior work separately examines multilingual models’ English-oriented representations, loss in natural-language agent communication, and unequal language resources. This paper addresses their intersection by testing explicit repeated translation in agent handoffs against an implicit English pivot.
2 Related Work
- Prior research documents that multilingual models can pass through an English-dominant latent region, while cross-lingual benchmark gaps scale with linguistic distance.The paper also notes evidence that models not anchored to English may activate distinct latent languages.Source: §2 · p. 2
- Natural-language agent handoffs are a lossy communication baseline, with proposed alternatives including raw-embedding communication and state-delta encoding.Prior work attributes loss in text handoffs to token sampling and studies how erroneous information propagates across communication topologies.Source: §2 · p. 2
- NLP resources remain concentrated in a few high-resource languages, and an orchestration penalty that scales with distance from English would disproportionately disadvantage distant, lower-resource languages.Source: §2 · p. 3
- The paper isolates the combination of English-oriented latent representations and lossy natural-language handoffs by testing explicit repeated translation instead of an implicit, single-pass pivot.Earlier work had examined these factors separately, including alternative communication protocols for agent systems.Source: §2 · p. 3
3 Methodology
The methodology isolates the effect of English-pivot communication by comparing native-language and English-forced multi-agent pipelines that share the same extraction facts. It evaluates these conditions on multilingual question answering with paired measures of performance and statistical testing.
3 Methodology
- The controlled design compares single-agent, native multi-agent, and English-forced multi-agent settings to separate English-routing costs from general orchestration overhead.The native and English-forced conditions use the same extracted facts; the English-forced condition adds forward translation, English processing, and back-translation.Source: §3 · p. 3, §3.2 · p. 3, Figure 1 · p. 4
3.1 Data and Model
- The evaluation samples 300 XQuAD questions per language across Spanish, Arabic, Hindi, and Chinese and uses the multilingual Aya-23-8B model.The authors select a strong multilingual model to help distinguish pipeline bottlenecks from weakness in the base model.Source: §3.1 · p. 3
3.2 The Three Conditions
- Condition E is scored against native-language gold answers so its end-to-end user-facing cost includes forward translation, English reasoning, and back-translation.The figure contrasts direct native-language answering in Condition N with the English-pivot pathway in Condition E.Source: §3.2 · p. 4, Figure 1 · p. 4
3.3 Metrics and Statistical Procedure
- The English-Forcing Tax, Accuracy(N) − Accuracy(E), isolates the English-pivot cost, while the Total Orchestration Penalty compares each multi-agent condition with the single-agent baseline.EM and token-level F1 measure performance; the English-pivot pathway includes forward translation, English processing, and back-translation.Source: §3.3 · p. 4
- Paired binary Exact Match outcomes for Conditions N and E are tested with exact two-sided McNemar tests, with a Bonferroni correction for four language comparisons.chrF between the forward translation and an independent English gold answer is used to relate lexical overlap to final pipeline success.Source: §3.3 · pp. 4–5
3.4 Compute and Reproducibility
- Experiments ran sequentially on a consumer-grade MacBook Air using Apple MLX, and greedy decoding at temperature 0.0 ensured deterministic outputs for paired testing.Evaluating temperature variance and sampling stochasticity remains future work.Source: §3.4 · p. 5
4 Results
English-forced routing reduces accuracy relative to native multi-agent execution across all four languages, with larger EM penalties for languages more distant from English. The section separates this added routing cost from general multi-agent overhead and reports language-specific differences in token-level outcomes.
Typological Gradient and Statistical Significance
- EM penalties for English-forced routing were 13.0 points in Spanish, 16.7 in Arabic, 27.7 in Chinese, and 30.6 in Hindi; all N–E differences survived Bonferroni correction.The comparison isolates English-forced from native multi-agent execution.Source: Figure 2 · p. 5, §4 · pp. 5–6, Table 1 · p. 6
Discordant Pair Asymmetry
- Native-versus-English discordant pairs were strongly asymmetric, ranging from 66:27 in Spanish to 106:14 in Hindi, with native-only successes more frequent in every language.The reported pairs are Native correct/English wrong versus English correct/Native wrong.Source: §4 · p. 6
Results
- The English-Forcing Tax (N–E) isolates added accuracy loss from the baseline multi-agent overhead (S–N), separating English routing cost from orchestration cost.Figure 4 depicts the overall S–E drop as the sum of these two components.Source: §4 · p. 5, Figure 4 · p. 6
The Chinese F1 Divergence
- Chinese showed a 0.277 EM penalty but a 0.181 F1 penalty, while Hindi’s EM and F1 penalties were 0.306 and 0.279, respectively.The 9.6-point Chinese divergence indicates greater partial token overlap in failure cases; the section notes that this language-specific pattern needs further investigation.Source: §4 · pp. 6–7, Figure 5 · p. 7
5 Diagnostic Correlational Analysis of English-Reference Lexical Overlap
The analysis tests whether Agent A’s English-reference lexical overlap is associated with the final outcome of the English-forced pipeline. Lower overlap is associated with failure, but the diagnostic does not isolate translation errors from downstream reasoning errors.
Diagnostic Correlational Analysis
- Mean chrF was 83.90 for successful pipelines versus 56.97 for failures, with lower English-reference overlap strongly associated with failure.The analysis covers 1,200 Condition E executions and uses an independent English gold answer.Source: Figure 6 · p. 7, §5 · p. 7, Table 2 · p. 7
- The diagnostic compares Agent A’s English output with an English gold answer, then relates chrF to pipeline outcome; it is lexical overlap, not a strict translation-fidelity score.It does not fully isolate translation errors from downstream English reasoning errors.Source: §5 · p. 7
- The authors recommend that agent frameworks consider native-language routing so the lossy forward-translation step can be skipped.They state that this step may contribute to subsequent reasoning failures, while English reasoning optimization remains important.Source: §5 · pp. 7–8
6 Illustrative Failure Modes
The examined failures illustrate how English-forced handoffs can lose entities, nuance, or lexical fidelity that native-language routing preserves. Some paraphrase changes retain meaning but still incur strict Exact Match penalties.
6 Illustrative Failure Modes
- A mistranslated proper noun, “Kony Ealy” becoming “Be Like Eli,” led the English-routed Answerer to hallucinate “Eli Manning.”The native-language condition produced the correct answer in this example.Source: §6 · p. 8
- English routing can prompt semantic drift: “Revolutionary Civil Disobedience” was overcompressed to “Revolution.”The native-language condition preserved context in this illustrative case.Source: §6 · p. 8
- Terms without direct English equivalents, including local administrative terminology, may be dropped, sacrificing context retained by native routing.Source: §6 · p. 8
- Paraphrases such as “Sierra Freeway” to “Sierra Highway” can preserve the concept yet incur strict Exact Match penalties.The passage distinguishes lexical noise from a complete loss of semantic meaning.Source: §6 · p. 8
7 Discussion
English-forced handoffs carry a translation cost compared with native-language routing. The discussion argues that native-language orchestration is important, particularly for typologically distant languages.
7 Discussion
- Up to 30.6 percentage points of Exact Match accuracy are sacrificed by English-forced handoffs versus the native-language multi-agent baseline, with larger penalties as typological distance from English increases.Source: §7 · p. 8
- Degraded forward translation is strongly associated with pipeline failure, so optimizing English reasoning alone may not be sufficient.The discussion recommends that agent frameworks support native-language routing throughout orchestration.Source: §7 · p. 8
- The translation cost is distinct from the diminishing value of adding agentic computation evaluated by Agrawal et al. (2026).Source: §7 · p. 8
8 Conclusion
The paper evaluates controlled multi-agent extraction-answer pipelines across four typologically diverse languages. It reports an English-Forcing Tax and connects reduced lexical overlap with pipeline failure.
8 Conclusion
- For typologically distant languages such as Hindi and Chinese, English-forced orchestration sacrifices up to 30.6 percentage points of Exact Match accuracy versus native-language orchestration.Source: §8 · p. 8
- The controlled two-agent pipeline is evaluated across four typologically diverse languages to isolate the English-internals default's architectural contribution.Source: §8 · p. 8
- Pipeline failure is strongly related to reduced English-reference lexical overlap, consistent with translation loss contributing to the performance drop.Source: §8 · p. 8
- The authors recommend that agent frameworks move beyond English-internal routing and support native-language inter-agent communication.Source: §8 · p. 8
Limitations
The evaluation uses a single multilingual model and zero-shot prompting, and its qualitative failure cases are illustrative rather than a validated taxonomy. The authors identify model, scale, and chain-depth coverage as future work.
Limitations
- Quantization may affect the observed tax, but its direction cannot be determined without an unquantized control.The evaluation also uses an 8-bit quantized Aya-23-8B version for consumer-grade edge deployment.Source: Limitations · p. 9
- The pipeline uses zero-shot prompting; few-shot examples or advanced prompt engineering may improve lexical overlap, but mitigation strategies are left for future work.Source: Limitations · p. 9
- The 20 presented failure cases are qualitative illustrations, not a statistically validated taxonomy.Source: Limitations · p. 9
- The study uses one 8B multilingual model; larger unquantized models may exhibit different failure modes.Source: Limitations · p. 9
- Further work should evaluate other model families, parameter scales, and agent chains deeper than three hops.Source: Limitations · p. 9
A Appendix A: Prompt Templates
Appendix A supplies reproducible prompt templates for the single-agent baseline and both multi-agent conditions, with target-language variables populated from aligned XQuAD examples.
A Appendix A: Prompt Templates
- The templates dynamically set {lang_name} and populate {context}, {question}, and {english_question} from aligned XQuAD examples.The {english_question} variable is the corresponding English question for the same example.Source: A Appendix A: Prompt Templates · p. 11
- The Single-Agent baseline answers directly in the target language from the passage and question.Source: A Appendix A: Prompt Templates · p. 11
- In Condition N, Agent A extracts relevant facts in the target language and Agent B answers using only those facts and the target-language question.Source: A Appendix A: Prompt Templates · p. 11
- In Condition E, Agent A’s output is translated into English, Agent B answers from English facts and question, and Agent C back-translates the answer.The native-language facts produced by Agent A are reused in both Conditions N and E.Source: A Appendix A: Prompt Templates · p. 11