PaperlayerView exact revision on arXiv

Source-linked AI summary

Quality-Aware Self-Correcting Speech Translation on an Edge Device

Zubair Ajmal Farooq, Diptesh Kanojia

arXiv:2610.07545v1cs.CL

TL;DR

Offline speech translation on a 4 GB edge device must address weak translations without relying on cloud services or retraining. This paper uses QE-triggered correction with three candidate-selection methods; MBR improves reported metrics, while QE functions as a gate rather than a ranker. The evaluation is limited to English-Spanish written benchmark data, with additional language pairs demonstrated but not evaluated.

  • Problem

    Cloud-based speech translation exposes audio, transcripts, and intent, while small edge-compatible cascaded models are susceptible to compounding errors; the paper asks whether an edge device can detect and correct weak translations fully offline without retraining.

    Source: §1 · p. 1

  • Method

    A Jetson Nano system uses lightweight QE to trigger a second-pass correction over an N-best pool, comparing QE reranking, MBR, and constrained beam search.

    Source: §1 · p. 1

  • Results

    MBR self-correction significantly improves lexical metrics and COMET at N=3; QE reranking yields no significant gains, while constrained beam search is structurally unsuited to correcting flagged output.

    Source: §1 · p. 1, §8 · p. 5

  • Takeaways & Limitations

    QE can serve as a trigger without ranking candidates, with memory-footprint implications for edge deployment; the paper identifies RAM as the principal deployment obstacle.

    Source: §1 · p. 1, §8 · p. 5

    Quantitative evaluation is limited to English-Spanish FLORES-200 written text in controlled domains, and performance on spontaneous speech, disfluencies, code-switching, or noisy audio is unmeasured.

    Source: §8 · p. 5

Abstract

from arXiv · show

We present a fully offline speech-to-speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper-tiny ASR feeds an Opus-MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary-pass correction when confidence falls below a pre-defined threshold $\tau$. We compare three correction methods: QE reranking (M1), Minimum Bayes-Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES-200 sentences (English-Spanish), M2 at $\tau=0.90$ produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1$\to$M2) does not hurt quality and frees 680 MB from the critical path. Using a gain-to-edit ratio adapted from the post-editing-effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six language pairs.

1 Introduction

The paper presents fully offline speech translation with QE-triggered self-correction on a 4 GB edge device. It reports significant MBR gains and argues QE is more useful as a trigger than as a candidate ranker.

1 Introduction

  • The system runs fully offline on a Jetson Nano with 4 GB of RAM and triggers a second correction pass when QE confidence is low.The system uses an N-best candidate pool and is evaluated across a 4×3 grid of thresholds and beam sizes.Source: §1 · p. 1
  • Minimum Bayes-Risk self-correction yields statistically significant improvements on BLEU, ChrF, and COMET.The paper evaluates 1,012 FLORES-200 sentences.Source: §1 · p. 1
  • QE serves as a reliable trigger but an unreliable candidate ranker, with implications for edge-device memory use.The paper reports that removing QE from candidate selection does not hurt quality and frees 680 MB from the critical path.Source: §1 · p. 1
  • The adapted gain-to-edit ratio connects per-sentence COMET gains to edit distance to examine lexical-semantic trade-offs across candidate-pool sizes.The paper reports more surgical corrections with better semantic adequacy for smaller pools, while larger pools maximise lexical reward.Source: §1 · p. 1
  • A single-load architecture brings steady-state latency below 30 seconds, while model reload dominates correction overhead on this hardware.The paper reports that a single-load configuration keeps all four models resident within the Nano’s 4 GB ceiling.Source: §1 · p. 1

2 Related Work

The related work covers on-device translation, reference-free quality estimation, translation reranking and constraint methods, and edit-distance-based effort measures. The paper positions its contribution as applying these ideas to offline, edge-device self-correction.

2 Related Work

  • End-to-end on-device speech translation has typically required server-class hardware or compressed cascades (Radford et al., 2023), while on-device quality control for composed pipelines is rare.The work targets a small publicly available NVIDIA edge device with 4 GB unified memory.Source: §2 · pp. 1–2
  • Reference-free QE estimates translation quality without parallel references; multilingual BERT cosine similarity is a lightweight signal, while COMET-Kiwi is larger but better correlated with human judgment (Specia et al., 2018; Rei et al., 2022b).The authors adopt QE as a gate even where it may fail as a ranker.Source: §2 · p. 2
  • Prior methods include MBR peer-consensus selection, QE-based reranking (Fernandes et al., 2022), and constrained decoding (Post and Vilar, 2018).The paper applies all three to self-correction and compares them with bootstrap testing.Source: §2 · p. 2
  • The gain-to-edit ratio adapts post-editing-effort measures to automatic self-correction by measuring COMET gain per edit distance between two translations.Earlier effort measures include TER, HTER, post-edit distance, and time-to-edit.Source: §2 · p. 2

3 System

The system cascades four components selected for their memory footprint under a 4 GB ceiling. It also describes a single-load memory configuration and demonstrates operation across six language pairs, while restricting quantitative evaluation to English-Spanish.

3 System

  • The four-component pipeline uses Whisper-tiny for English ASR, Opus-MT for English-to-Spanish translation, multilingual BERT cosine similarity for QE, and eSpeak-ng for Spanish speech output.First-pass translation uses greedy decoding; second-pass correction uses beam search with N ∈ {3, 5, 10}.Source: §3 · p. 2
  • Keeping all four models resident uses about 3.6 GB under the 4 GB ceiling and reduces latency by avoiding repeated model loading.Under sequential loading, each stage incurs a load/unload cost that dwarfs inference.Source: §3 · p. 2
  • The architecture supports six language pairs by swapping the Opus-MT model and TTS voice, but quantitative evaluation is restricted to English-Spanish.Live demonstrations confirmed operation across the supported pairs at comparable steady-state latency.Source: §3 · p. 2

4 Self-Correction Methods

The system generates alternative translations and selects one using QE reranking, MBR peer consensus, or constrained beam search. These methods differ in how they score or constrain the candidates.

4 Self-Correction Methods

  • The system generates N alternative translations and selects one using one of three correction methods.Source: §4 · p. 2
  • M1 selects the candidate with the highest mBERT cosine score against the source, using the same QE measure as the gate.Source: §4 · p. 2
  • M2 selects the candidate with the highest mean ChrF similarity to the other N−1 candidates and does not consult the source or QE model after generation.Source: §4 · p. 2
  • M3 extracts content words longer than three characters from the greedy output as soft anchors, then boosts hypotheses containing them and reranks candidates by mBERT QE.Source: §4 · p. 3

5 Experimental Setup

The experiments evaluate English-to-Spanish speech translation on FLORES-200 using a grid of correction thresholds and beam sizes, with greedy decoding as the baseline. They assess translation quality with three metrics and paired-bootstrap significance tests on a Jetson Nano.

Data

  • 1,012 English-to-Spanish FLORES-200 devtest sentences form the translation set, balanced across topical domains.English FLEURS audio with paired transcripts is used for the ASR component.Source: §5 · p. 3

Grid

  • The study sweeps four QE thresholds and three beam sizes across all correction methods, yielding 36 configurations against one greedy baseline.The baseline uses N=1 with no correction.Source: §5 · p. 3

Metrics

  • SacreBLEU, ChrF, and COMET-22 measure quality, with paired bootstrap resampling over 1,000 samples testing improvement over baseline.The study reports 95% confidence intervals and one-tailed p-values.Source: §5 · p. 3

Hardware

  • ASR, MT, and QE run on a Jetson Nano with 4 GB RAM; COMET evaluation runs on Google Colab CPU because the model exceeds the Nano’s available memory.The same model weights and hypothesis files are used in both environments.Source: §5 · p. 3

6 Results

Across the tested correction methods, MBR improves translation quality, while QE reranking does not significantly change it and constrained beam search performs worse than baseline. Results also show different beam-pool-size trade-offs, more efficient sentence-level corrections at smaller pools, and a memory benefit from using QE as a trigger rather than a ranker.

6.1 Aggregate Quality

  • At τ=0.90, M2 significantly improves all three metrics, M1 shows no significant change, and M3 is significantly worse than baseline.Table 1 reports the strongest configuration for each method; the M2 COMET result at N=10 is not individually significant.Source: §6.1 · p. 3, Table 1 · p. 3

6.2 N-Trade-off: Lexical vs Semantic

  • +0.67 BLEU and +0.51 ChrF at N=10 are significant (p<0.001), while COMET peaks at N=3 (+0.0020, p=0.002).The full grid shows M2 outperforming M1 and M3 whenever correction fires at τ ≥0.85.Source: §6.2 · p. 3, Table 2 · p. 3

6.3 Gain-to-Edit Ratio

  • At N=3, M2 has the highest median GER; N=5 is lowest, while N=10 has a negative mean (−0.0019) but positive median (+0.0048).The account attributes the N=10 asymmetry to a long tail of large, low-gain edits; GER distinguishes targeted small edits from larger rewrites.Source: §6.3 · p. 4, Table 3 · p. 4

6.4 QE as Gate, Not Ranker

  • Removing QE from candidate selection while retaining it as the trigger does not degrade quality, and M2 outperforms M1 on every metric.Unloading QE during MBR selection frees about 680 MB of resident memory; the authors suggest separating trigger and ranking roles in similar cascaded MT pipelines.Source: §6.4 · p. 4

6.5 M3 Failure

  • M3 underperforms baseline at every tested N (p>0.99), which the authors attribute to preserving anchors from the low-confidence first-pass output.They identify terminology enforcement and named-entity preservation as possible alternative roles for constrained beam search.Source: §6.5 · p. 4

7 Latency Analysis

On the Jetson Nano, model reload dominates sequential-load latency, while keeping models resident lowers latency within the device’s 4 GB memory ceiling. Removing reload also reduces MBR correction overhead.

7 Latency Analysis

  • Resident loading uses 3.6 GB and brings per-utterance latency to approximately 41 s, versus approximately 82 s under sequential loading.The single-load measurement is for one approximately 10 s en-es utterance, including first-utterance load amortisation.Source: §7 · p. 5, Figure 2 · p. 5
  • 23.9 s of MBR correction overhead falls to 13.2 s when model reload is removed from the critical path.Under sequential loading, the load/unload cycle around MT Pass 2—not beam search—dominates correction overhead.Source: §7 · p. 5
  • The 4 GB ceiling forced sequential loading, making latency RAM-bound rather than compute-bound in the original architecture.The authors state that a modestly larger memory budget would allow near-real-time latency without changing the algorithms.Source: §7 · p. 5

8 Discussion and Limitations

The discussion concludes that QE-triggered MBR correction improves quality, while QE is more useful as a trigger than a ranker. The evaluation is limited to English–Spanish written-text data and does not measure spontaneous or noisy speech.

8 Discussion and Limitations

  • QE-triggered MBR improves lexical metrics and COMET at N=3, while the QE model is useful as a trigger but not as a ranker.The discussion also identifies a lexical-versus-semantic trade-off across pool sizes, analyzed through sentence-level GER, and reports CBS as structurally unsuited to correction on flagged output.Source: §8 · p. 5
  • Evaluation covers English–Spanish FLORES-200 devtest, not spontaneous speech with disfluencies, code-switching, or noisy acoustics.The passage also notes that the QE signal is multilingual BERT cosine similarity, a weak proxy for semantic adequacy.Source: §8 · p. 5
  • The conclusion identifies RAM, rather than self-correction algorithm cost, as the principal obstacle to conversational on-device speech translation.It describes the offline system as requiring no retraining and using MBR over a small candidate pool with QE as a gate.Source: §8 · p. 5

A Full Grid Search results

The full grid search spans three correction methods, four QE thresholds, and three beam sizes on 1,012 FLORES-200 sentences. The supplied passages identify the internal BLEU and ChrF baselines and explain the table’s delta notation.

A Full Grid Search results

Loading 2610.07545v1…