PaperlayerView exact revision on arXiv

Source-linked AI summary

Bilingual Language Models Capture Cross-Linguistic Influence in Second-Language Reading

Abderrahmane Issam, Irene E. Winther, Stefan L. Frank

arXiv:2610.31620v1cs.CL

TL;DR

Monolingual surprisal cannot capture L1-driven influences that may shape L2 reading, leaving the predictive relevance of bilingual-model transfer an open question. The paper evaluates bilingual models against English eye-tracking data and matched and mismatched controls, finding that the L1-matched advantage is concentrated in late-stage processing and varies across readers and scripts.

  • Problem

    It remains an open question whether bilingual-model cross-lingual representations align with L1-driven influence during L2 reading, which monolingual models cannot capture.

    Source: §1 · p. 1

  • Method

    The study evaluates bilingual models on English eye-tracking data and compares L1-matched models with monolingual and L1-mismatched controls.

    Source: §1 · p. 1, §3.2.1 · p. 4, §5 · p. 7

  • Results

    The L1-matched predictive advantage is concentrated in late-stage measures, varies with script overlap, and peaks among intermediate-proficiency readers.

    Source: §5 · pp. 7–8

  • Takeaways & Limitations

    Bilingual models capture aspects of L1-specific predictive biases that mirror human non-native reading, particularly in late-stage processing.

    Source: §5 · p. 8

    The evaluation uses one eye-tracking corpus and English as the target L2, leaving generalization to typologically different target languages open.

    Source: Limitations · p. 9

Abstract

from arXiv · show

While monolingual language model (LM) surprisal is widely established as a predictor of reading times, it cannot capture the non-native (L2) reading dynamics that are shaped by the reader's first language (L1). In this work, we investigate whether the cross-lingual transfer within bilingual models aligns with human L1-driven biases during L2 reading. We evaluate bilingual models on eye-tracking data of readers from 12 diverse L1 backgrounds reading English texts. To isolate L1-specific transfer, we compare bilingual models trained on an L1 that matches the readers' L1 background to both a monolingual English baseline and bilingual models trained on other L1s. We demonstrate that matching the readers' L1 improves the prediction of L2 reading times. This predictive advantage is concentrated in late-stage eye-tracking measures, modulated by L1-L2 script overlap, and peaks among intermediate-proficiency readers. These findings provide evidence that bilingual LMs capture aspects of cross-linguistic influence, and that such influence is present in L2 reading behavior.

1 Introduction

The paper asks whether bilingual-model transfer reflects L1-driven influence during L2 reading, which monolingual models cannot capture. It tests this with English eye-tracking data and finds an L1-matched advantage concentrated in later processing.

1 Introduction

  • It remains less clear whether bilingual-model surprisal predicts non-native reading behavior than whether monolingual surprisal predicts native reading times.The supplied background describes substantial work on monolingual surprisal and L1 reading, but limited knowledge about bilingual models and L2 reading.Source: §1 · p. 1
  • Because readers’ L1 may shape L2 reading, a monolingual model trained only on the target language cannot capture that influence.The paper frames cross-linguistic influence as a candidate source of differences between native and non-native reading.Source: §1 · p. 1
  • The study evaluates bilingual GPT-2 surprisal on English eye-tracking data from readers with 12 L1 backgrounds, comparing matched models with monolingual and mismatched controls.It varies when English is introduced during bilingual training and examines four eye-tracking measures spanning early and later processing.Source: §1 · pp. 1–2
  • L1-matched bilingual models improve L2 reading-time prediction, chiefly in late-stage measures; the advantage varies with script overlap and peaks at intermediate proficiency.The authors conclude that bilingual models capture aspects of cross-linguistic influence present in human L2 reading.Source: §1 · p. 2

2 Related Works

Prior work has modeled bilingual exposure in several ways and studied transfer effects, but evidence on bilingual models predicting L2 reading times remains limited. This study addresses gaps in prior evaluations through broader comparisons and measures.

2.1 Bilingual Models for L2 Acquisition

  • Prior bilingual-acquisition studies vary exposure balance, introduce L2 sequentially, freeze L1 representations, or use specialized objectives; this study adopts sequential training to predict L2 reading times.Prior work examined syntactic transfer, priming, and L1 influence on L2 reading times.Source: §2.1 · p. 2

2.2 Evaluating LMs on L2 Reading Times

  • Most L2 reading-time evaluations use monolingual surprisal, while earlier bilingual-model studies lacked controls or broad measures that isolate L1-specific transfer.This study uses unfrozen models, tests matched surprisal beyond mismatched surprisal in the same regression, and evaluates four measures across 12 L1s, proficiency, and script overlap.Source: §2.2 · p. 2

3 Methods

The methods train bilingual models with varying L1 exposure and evaluate surprisal against MECO eye-tracking data using two complementary model comparisons. A further analysis tests whether predictive advantage varies with reader proficiency.

3.1 Bilingual LM Training

  • Bilingual GPT-2 models receive sequential L1 and L2 training, with Step of Exposure varying when L2 is introduced while total training steps remain fixed.The four exposure stages begin bilingual training at steps 0, 16k, 32k, or 48k; an English-only baseline is also trained for each language pair.Source: §3.1 · p. 3

3.2 Evaluation Framework

  • Models are evaluated on MECO English reading data with four eye-tracking measures, using word surprisal in mixed-effects regressions with lexical and positional baseline covariates.The measures are first fixation duration, gaze duration, go-past time, and total fixation duration; MECO subsets are kept by institution, with native English as a control.Source: §3.2 · pp. 3–4

3.2.1 Model Comparisons

  • The comparisons test bilingual against monolingual surprisal using ΔR², then matched against mismatched bilingual surprisal using likelihood differences and nested-model tests.The nested comparison estimates variance uniquely added by matched surprisal while holding baseline covariates and random effects constant.Source: §3.2.1 · p. 4

3.3 Proficiency Modulation Analysis

  • Reader-specific bilingual advantage is modeled against mean-centered LexTALE proficiency using weighted linear or quadratic regressions, with L1-subset controls; the quadratic term remains significant.The quadratic specification is compared with the linear model, and the refitted model uses L1-subset fixed effects and clustered standard errors.Source: §3.3 · pp. 4–5

3.2 Evaluation Framework

  • Except for first fixation duration, bilingual surprisal provides greater predictive power for L2 than L1 readers.The figure describes predictive power across four reading measures as a function of Step of Exposure.Source: §3.3 · p. 5

4 Results and Analysis

Matched bilingual models provide more unique predictive information than mismatched models for later L2 reading measures, with the advantage varying by script and reader proficiency. Bilingual predictive gains are limited in early measures and peak among intermediate-proficiency readers.

4.1 Are L2 Models Better at Predicting L2 Data than Monolingual Models?

  • Bilingual surprisal provides no advantage for first fixation, while gaze duration and go-past time gains increase with SoE and total fixation time gains remain positive.Native English controls stay close to zero and below the L2 mean on gaze duration, go-past time, and total fixation duration.Source: §4.1 · p. 5

4.2 Are Matched Models Better Than Mismatched Models?

  • Except in first fixation, matched models contribute more unique log-likelihood and achieve higher significance rates than mismatched models across later reading measures.The nested comparisons indicate that matched models explain significantly more unique variance during late-stage reading.Source: §4.2 · pp. 5–6, Table 1 · p. 6

4.3 The Role of Script Overlap

  • Latin and Non-Latin groups diverge on early measures, but both groups perform comparably to or better than the monolingual baseline on later measures.For gaze duration, Latin models remain positive while Non-Latin models hover near or below the baseline; Non-Latin go-past advantage rises steadily with SoE.Source: §4.3 · p. 6, Figure 3 · p. 7

4.4 Effect of Reader Proficiency

  • The bilingual advantage follows an inverted-U for late measures, peaking at 65–80% LexTALE and descending toward zero above 90%.Early lexical access measures remain linear; the late-measure trajectory is replicated with a composite proficiency index.Source: §4.4 · p. 7

5 Discussion and Conclusion

L1-matched bilingual models capture aspects of L1-specific effects on L2 reading, with their predictive advantage concentrated in later processing. That advantage varies with proficiency, writing-system overlap, and measure-specific exposure effects.

5 Discussion and Conclusion

  • Across comparisons with monolingual and L1-mismatched models, bilingual surprisal captures aspects of L1-specific effects, with its advantage absent in first fixation, emerging in gaze, and strongest in late measures.The two evaluation frameworks yield consistent results.Source: §5 · p. 7
  • Late-measure bilingual advantage follows an inverted-U across proficiency, peaking among intermediate readers at 65–80% LexTALE.Early measures show negligible non-linearity; the late measures are Go-Past and Total Fixation.Source: Figure 4 · p. 8, §5 · p. 8
  • Step of Exposure has no uniform effect across measures, although go-past predictability consistently rises as SoE increases.The authors suggest that prolonged L1 training may affect prediction differently across L1–L2 pairs.Source: §5 · p. 8
  • In gaze duration, only Latin-L1 models show an advantage, while both script groups compete with or outperform the monolingual baseline in late measures.The proposed account is that shared scripts allow earlier orthographic co-activation, whereas differing scripts delay effects; the authors say these mechanisms are not definitively confirmed.Source: §5 · p. 8
  • Go-past time most consistently aligns with L1-specific processing because it tracks time to detect and resolve breakdowns, including regressive movements linked to L1–L2 conflicts.The authors caution against relying solely on early reading measures.Source: §5 · p. 8
  • The findings support evaluating L2 reading across its temporal complexity rather than assuming a uniform bilingual-model advantage.The alignment is concentrated in late-stage structural processing and peaks among intermediate learners.Source: §5 · p. 8

Limitations

The evaluation has boundaries in target language, corpus, script-based analysis, and proficiency measurement; broader generalization remains open. The appendices describe the MECO subsets and GPT-2 model settings.

Limitations

  • Findings are limited to English as the target L2 and one eye-tracking corpus, leaving generalization to typologically different target languages open.Regional language varieties may also differ from the standard, general-domain L1 training corpora, potentially adding noise and explaining some subset-level performance variation.Source: Limitations · p. 9
  • Script overlap is only one axis of variation; word order and morphological similarity may also affect L2 processing and remain to be investigated.Source: Limitations · p. 9
  • LexTALE is a lexical proficiency measure, and even the MECO-derived composite primarily indexes lower-level skills, leaving higher-level comprehension and syntactic awareness unresolved.The authors report robustness with a PCA-derived composite of MECO proficiency metrics, but these still mainly measure vocabulary size and decoding efficiency.Source: Limitations · p. 9

A MECO Subsets and Languages

B Hyperparameters

C Bilingual Model Surprisal Correlation

English surprisal estimates from the L1 bilingual models are highly correlated, but their mean correlation declines slightly as L1 exposure increases.

D.1 Isolated effects of Bilingual and Monolingual Surprisal

Monolingual and bilingual surprisal show similar proficiency trajectories, while bilingual models diverge slightly above the monolingual baseline in late reading measures. The subtraction analysis isolates this advantage.

  • Both surprisal models follow a similar inverted-U trajectory across proficiency, peaking for intermediate readers, consistent with greater reliance on context-dependent expectations.Source: §D.1 · pp. 13–14
  • Figure 6 shows explanatory power over the surface-covariate baseline across LexTALE proficiency; both model types exhibit an inverted-U relationship.Source: Figure 6 · p. 14
  • In go-past time and total fixation duration, bilingual surprisal diverges from and peaks slightly above the monolingual baseline.The difference between bilingual and monolingual explanatory power isolates the cross-linguistic transfer effect, which also scales non-linearly with proficiency.Source: §D.1 · pp. 13–14

D.2 Quadratic Relationship Evaluation

The proficiency relationship in the bilingual advantage is non-linear for late reading measures at longer SoE stages, but not statistically supported for early measures.

  • For go-past time and total fixation duration, quadratic models fit better for SoE ≥16k (pquad < 0.05, ∆AIC > 2.0), with negative βquad coefficients.Table 4 marks exposure stages where both criteria favor the quadratic model.Source: §D.2 · p. 14, Table 4 · p. 14
  • In go-past time, ∆AIC rises from +0.6 at SoE 0 to +9.5 at SoE 48k, supporting a stronger quadratic fit with longer L1 training.Source: §D.2 · p. 14
  • For first fixation duration and gaze duration, the quadratic term is not statistically significant, and negative ∆AIC generally favors the linear baseline.Source: §D.2 · p. 14

D.3 Robustness to Composite Proficiency (PCA)

Composite proficiency analyses largely reproduce the LexTALE pattern: the bilingual advantage peaks at intermediate proficiency in late-stage measures, especially go-past time. Early measures show no significant quadratic proficiency effect, while total fixation duration is less robust under L1-subset controls.

D.3 Robustness to Composite Proficiency (PCA)

  • 52.5% of assessment variance is explained by PC1, a composite from five standardized proficiency assessments.The assessments are LexTALE, vocabulary knowledge, spelling, and TOWRE-2 word and nonword tests; Figure 7 models the advantage against composite proficiency across four reading measures.Source: §D.3 · p. 15, Figure 7 · p. 15
  • The inverted-U advantage among intermediate-proficiency readers generalizes across proficiency assessments, particularly for go-past time.The go-past trajectory is significant across all SoE; total fixation duration shows the same qualitative curvature but reaches significance only at SoE=16k under strict L1-subset controls.Source: §D.3 · p. 15, Figure 7 · p. 15
  • Early measures show no significant quadratic proficiency term across SoE, while go-past time retains a significant inverted-U trajectory.For total fixation duration, the inverted-U pattern is qualitatively similar, but its significance is attenuated under strict L1-subset controls.Source: §D.3 · p. 15
Loading 2610.31620v1…