PaperlayerView exact revision on arXiv

Source-linked AI summary

AS$^2$D: Accelerating On-Demand Audio Understanding on Mobile Devices

Yunzhe Li, Kyoungjun Park, Hongzi Zhu, Lili Qiu

arXiv:2609.37617v1cs.SDcs.AI

TL;DR

Conventional speculative decoding ties drafting to the target’s evolving verified prefix, serializing work. AS2D lets an audio-conditioned drafter advance independently while the target verifies and corrects ready candidates. Across phones, it improves pooled ASR throughput over target-only decoding and achieves up to 78% higher throughput in native 7B on-demand execution.

  • Problem

    Conventional speculative decoding makes the drafter wait for the target’s evolving accepted or corrected prefix, coupling drafting and verification.

    Source: §1 · p. 1

  • Method

    AS2D uses audio- and instruction-conditioned drafting independent of target-prefix updates while the target verifies ready candidates or decodes alone and determines committed output.

    Source: §1 · p. 2, §4 · p. 4

  • Results

    Across four phones, AS2D improves pooled ASR throughput by 42–76% over target-only decoding, and native on-demand execution with a 7B target achieves up to 78% higher throughput than target-only.

    Source: §1 · p. 3, §6 · p. 9

  • Takeaways & Limitations

    Source-conditioned audio generation can relax the target-prefix dependence of speculative drafting while retaining target-side verification and correction.

    Source: §1 · p. 2, §6 · p. 9

    The study uses existing speech datasets and prerecorded audio, and does not address errors, bias, or harmful outputs of the underlying models.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Speculative decoding accelerates autoregressive generation by using a smaller drafter to propose tokens for batched verification by a larger target. However, conventional speculative decoding couples drafting to the target's evolving verified prefix, serializing drafting and verification. We ask whether this dependency is necessary for source-conditioned generation. Our key observation is that, for audio language models, the input audio and user request can provide useful speculative candidates without following the target's evolving text prefix. We propose AS$^2$D (Audio Speculative Speculative Decoding), which enables target-decoupled drafting: an audio-conditioned drafter follows its own generation history while the target independently verifies and corrects ready candidates. Without usable candidates, the target advances alone. Thus, target feedback determines which candidates are committed but no longer determines when the drafter can make progress, enabling drafting and verification to proceed concurrently while retaining target-side verification and correction. We implement AS$^2$D in MNN for Android and evaluate two target models across four phones, seven datasets, and three tasks covering 12.2 hours of audio. Across four phones, AS$^2$D improves pooled ASR throughput by 42-76% over target-only decoding, while only 5.7% of evaluation windows are slower than target-only, compared with 58.1-63.0% for speculative baselines. For ASR, AS$^2$D reaches 97.33-98.20% of a hindsight per-window oracle's pooled throughput over the evaluated drafter/budget catalog. Native on-demand execution with a 7B target achieves up to 78% higher throughput than target-only. These results show that source-conditioned audio generation can relax the conventional dependence of speculative drafting on the target's evolving output prefix, exposing substantial parallelism for efficient inference.

1 INTRODUCTION

Conventional speculative decoding still makes drafting wait for the target’s evolving verified prefix. AS2D uses audio-conditioned, target-decoupled drafting to overlap candidate generation and verification while preserving target-side control of committed output; its evaluation reports faster decoding across mobile settings.

Motivation

  • Conventional speculative decoding couples drafting to the target’s evolving accepted or corrected prefix, leaving drafting and verification sequentially dependent.Batched verification can save target work, but serial drafting can consume those savings on phones.Source: §1 · p. 1, Figure 1 · p. 2
  • Audio and the instruction can supply useful future candidates without waiting for the target’s latest verified text prefix.Direct audio processing preserves signal information such as intonation, stress, and emotional cues that transcription can discard.Source: §1 · pp. 1–2

Approach

  • AS2D lets an audio-conditioned drafter follow its own history while the target verifies ready candidates or decodes alone, retaining target-side correction and commitment.Target feedback determines which candidates are committed, but no longer determines when the drafter can progress.Source: §1 · pp. 2–3

Evaluation

  • 42–76% higher pooled ASR throughput than target-only decoding, with only 5.7% of windows slower versus 58.1–63.0% for speculative baselines.Across four phones, AS2D also reaches 97.33–98.20% of a hindsight per-window oracle’s pooled throughput over the evaluated drafter/budget catalog.Source: §1 · p. 3
  • The Android evaluation covers two target models, four phones, seven datasets, and recognition, translation, and spoken question answering across 12.2 hours of audio.Native on-demand execution with a 7B target achieves up to 78% higher throughput than target-only decoding.Source: §1 · p. 3

2 RELATED WORK

Prior work prepares audio context, accelerates speculative speech decoding, or overlaps drafting and verification. AS2D differs by letting an audio-conditioned producer advance independently of target-prefix updates while retaining target verification and correction.

On-demand audio understanding

  • Audio-memory systems retain prepared source context for later queries, while AS2D complements them by accelerating response generation under mobile resource constraints.MiniCPM-o 2.6 separates language-model prefill from response generation, and OmniMem compresses cached audio-visual memory; both leave sequential decoding on the response path.Source: §2 · p. 3

Speculative speech decoding

  • SpecASR uses acoustic alignment and local candidate reuse but regenerates drafts from verified text; AS2D instead lets its audio-conditioned producer advance independently.AS2D retains target verification and correction across recognition, translation, and question answering.Source: §2 · p. 3

Asynchronous and mobile execution

  • Prior asynchronous methods overlap work through parallel target execution, cached continuations, or specialized mobile hardware, whereas AS2D uses audio and instruction for target-decoupled candidates.On a single phone, extra computation and speculative state compete with verification for limited resources.Source: §2 · p. 3

3 PROBLEM FORMULATION

The formulation treats on-demand audio inference as choosing a decoding scheme for a fixed phone and request distribution. The scheme should minimize response latency while preserving target output and respecting memory and any required tail-latency limit.

3 PROBLEM FORMULATION

  • A request is an audio view and instruction, with fixed audio preparation, prompt construction, state handling, and termination under protocol S.The target’s completed audio-derived KV state is retained for generation.Source: §3 · pp. 3–4
  • A decoding scheme specifies candidate generation, verification, and scheduling subject to input availability, candidate readiness, backend support, output limits, and accepted-prefix restoration.Completion time measured from request arrival defines response latency.Source: §3 · p. 4
  • The optimization minimizes mean response latency while preserving the target output, satisfying the phone’s memory budget, and meeting any required latency-quantile limit.Post-prefill throughput isolates decoding; response latency includes work after request arrival.Source: §3 · p. 4

4 DESIGN OF AS2D

AS2D separates audio-conditioned candidate production from target verification and aligns reusable candidates after target corrections. Its potential speedup depends on hiding drafting work while supplying ready candidates that cover enough target output.

4.1 TARGET-DECOUPLED DRAFTING

  • Audio and instruction condition a fixed drafter independently of target-prefix updates, while target verification uses retained audio KV and committed context.A controlled ASR probe found unchanged acceptance in 72.3% of 480 paired blocks after preceding transcript context was omitted.Source: §4.1 · p. 5, §4 · p. 5
  • The producer publishes ordered token batches and continues its own generation while the target verifies earlier candidates; a separate cursor tracks usable continuations.If no usable continuation is ready, the target decodes alone.Source: §4.1 · p. 5

4.2 VERIFICATION AND ALIGNMENT

  • After target correction, local token matching moves the buffer cursor to a reusable continuation; the target verifies those candidates under its updated context.If no match is found, the target continues alone while the producer keeps generating.Source: §4.2 · p. 5, Figure 4 · p. 5

4.3 DEVICE-PROFILED CONFIGURATION

  • The device-profiled budget is selected offline from phone costs and development traces, then fixed during deployment while candidate readiness may narrow actual verification width.The selected budget is subject to memory and backend constraints.Source: §4 · p. 4, §4.3 · p. 6

4.4 WHEN DOES TARGET-DECOUPLED DRAFTING HELP?

  • Drafting accounts for 49.4–75.1% of native serial speculative-decoding time, and target decoupling can reduce idealized cost from D + V to max(D, V) + H.D is drafting, V is verification/decoding, and H is additional overhead.Source: §4.4 · p. 6
  • Speedup depends on drafting cost, candidate readiness, and useful candidate coverage; spoken QA has 84.7% acceptance but 28.3% coverage and improves throughput 8.0% over the strongest speculative baseline.High acceptance alone is insufficient when candidates cover little of the target output.Source: §4.4 · p. 6

5 EVALUATION

The evaluation tests AS2D on mobile devices across audio tasks, compares it with target-only and speculative baselines, and examines the sources and native performance of its speedups. AS2D improves pooled ASR throughput across phones, with additional gains in translation, spoken QA, and native on-demand execution.

5.1 EXPERIMENTAL SETUP

  • The study implements AS2D in MNN for Android and evaluates two target models across four phones, seven datasets, and three task families.The corpus covers 12.2 hours of audio and 97,953 target output tokens; comparisons include target-only, standard speculative decoding, and SpecASR.Source: §5.1 · pp. 6–7, §5.2 · p. 7

5.2 RQ1: PERFORMANCE ACROSS PHONES, MODELS, AND TASKS

  • 42–76% higher pooled ASR throughput than target-only across four phones, with AS2D at 27.92 TPS on K70 using budget 7.Figure 5 shows per-window TPS distributions for the same 688 complete-audio windows across methods; horizontal scales differ by phone.Source: Figure 5 · p. 7, §5.2 · p. 7
  • AS2D reaches 97.33–98.20% of the hindsight oracle’s overall pooled TPS across the four phones.The oracle selects a feasible drafter and fixed budget per window from the evaluated catalog, with zero selection cost.Source: §5.2 · p. 8, Table 2 · p. 8
  • Across translation and spoken-QA tasks, AS2D improves TPS by 17.4%, 31.2%, and 8.0% over the fastest measured baseline.The comparison uses 80 examples per task and a shared Omni-7B target; QA is a post-hoc subgroup requiring at least 16 useful target tokens.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 RQ2: SOURCES OF THE SPEEDUP

  • Target-decoupled drafting raises pooled TPS across four phones from 8.49 to 12.90, a 51.9% increase.This ablation measures the joint benefit of target-prefix independence and asynchronous overlap.Source: §5.3 · p. 8, Table 5 · p. 9
  • Replacing ASP’s budget rule raises K70 TPS by 9.0% when target-decoupled drafting is already present.The deployment uses a fixed budget throughout each phone’s test.Source: §5.3 · p. 9

5.4 RQ3: NATIVE ON-DEMAND ACCELERATION

  • Native on-demand execution improves K70 throughput over target-only by 26.7–78.0% across 19 request positions.On Xiaomi 14, AS2D improves pooled TPS by 28.2% over target-only and 8.5% over SD.Source: §5.4 · p. 9, Figure 7 · p. 9
  • On Redmi K70 Pro, continuous processing keeps pace with 4-second audio blocks over 600 seconds under the tested configuration.Mean block processing is 3.63 seconds; p95 completion lag is 4.68 seconds.Source: §5.4 · p. 9

6 CONCLUSION

The conclusion presents AS2D as target-decoupled drafting for source-conditioned audio generation, with measured gains in replay and native on-demand execution. The supplied material also states the evaluation’s scope, execution protocols, and a cross-corpus variation in candidate acceptance.

6 CONCLUSION

  • AS2D’s measured gains depend on candidate supply and alignment, and the conclusion identifies adaptive processor placement and scheduling and full-duplex interaction as future work.The Android evaluation spans two target models, four phones, seven datasets, and three tasks.Source: §6 · p. 9

AI USE STATEMENT

  • Generative AI tools assisted across research and manuscript tasks, while authors defined the questions and scope, directed and reviewed the assistance, and retained responsibility.The authors state that research and editorial decisions remained theirs.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper documents protocols, configurations, dataset selection, baselines, development splits, calibration, replay/native comparisons, and observed numerical discrepancies in appendices.Appendix F.6 retains the discrepancies and their implications.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

  • The work evaluates execution scheduling for pretrained audio models, not their errors, bias, or harmful outputs.The evaluation uses existing speech datasets and prerecorded audio.Source: ETHICS STATEMENT · p. 10

A.1 REQUESTS, AUDIO AVAILABILITY, AND REFERENCE EXECUTION

  • The on-demand reference fixes each request’s audio view at request arrival, even if more audio arrives before the response completes.Compared methods use the same protocol and target rather than changing the reference context.Source: §A.1 · p. 15
  • Audio KV reuse requires a valid prefix of the reference prompt with matching audio features, positions, and attention conventions.An instruction that changes earlier prompt tokens can require reconstruction.Source: §A.1 · p. 15

A.2 CONFIGURATION AND EXECUTION BOUNDARIES

  • The decoding scheme uses a fixed task-specific drafter and deployment budget, while candidate readiness can reduce the actual verification width.A proposal cap b allows up to b candidates after one known target root, so physical verification width is at most b + 1.Source: §A.2 · p. 15

A.3 RESPONSE LATENCY, THROUGHPUT, AND MEMORY

  • Post-prefill throughput isolates decoding cost and does not establish that a growing audio KV cache remains ready on a phone.The continuous-output deployment uses a different timing boundary from an on-demand request.Source: §A.3 · p. 16

B.2 DEVICE-SPECIFIC VERIFICATION PROFILES

The block profiles phone-specific verification costs and examines the costs and behavior of asynchronous audio proposals. It then specifies how candidate publication, target verification, state repair, and alignment operate.

B.2 DEVICE-SPECIFIC VERIFICATION PROFILES

  • Per-position savings fell from 60.1% to 50.0% as K increased from 4 to 5 on Xiaomi 11 Pro, but stayed near 64.5% on Redmi K70 Pro.The plotted values are verifier savings at profiled contexts, not full-request gains.Source: Figure 8 · p. 16, §B.2 · p. 17

B.3.1 NATIVE SERIAL DECODING BREAKDOWN

  • Drafting accounted for 49.4%, 59.1%, 75.1%, and 51.2% of serial decode time on Xiaomi 11 Pro, Redmi K70 Pro, Xiaomi 14, and Redmi Note 9 Pro, respectively.These are within-run measurements on one audio window, not a controlled processor-speed ranking or corpus-wide estimate.Source: §B.3.1 · p. 17

B.3.2 AUDIO PROPOSALS CAN RUN AHEAD

  • Removing preceding transcript context left acceptance unchanged in 347 of 480 pairs (72.3%), with a median difference of zero.Acceptance fell by more than 0.05 in 9.6% of pairs and rose by more than 0.05 in 15.6%; this does not imply unchanged output tokens or uniform acceptance across tasks.Source: §B.3.2 · pp. 17–18

B.4 NATIVE DRAFTING-COST CHECK

  • 14.58 TPS for target-only, 16.25 for Parakeet/K = 4, and 16.03 for SenseVoice/K = 4 were measured on one AMI window; the latter two gains were 11.5% and 9.9%.Increasing Parakeet to K = 8 raised median extra accepted tokens from 39 to 43 but lowered throughput to 15.66 TPS.Source: §B.4 · p. 18

C CANDIDATE PUBLICATION, VERIFICATION, AND CORRECTNESS

C.1 SOURCE RECORDS AND INCREMENTAL PUBLICATION

  • A source record fixes the request’s audio view and instruction; Qwen publishes immutable eight-token batches while verification reads a snapshot and advances its cursor independently.External Parakeet and SenseVoice experiments publish completed 2.56-second chunks atomically.Source: §C.1 · p. 18

C.2 VERIFICATION AND STATE REPAIR

  • Verification checks a known target root with up to b candidates, commits only the matching greedy prefix, repairs state after rejection, and takes a K = 1 step when no compatible candidate is ready.Physical width is also limited by candidate readiness and remaining output length.Source: §C.2 · pp. 18–19

C.3 CANDIDATE ALIGNMENT

  • After rejection, bounded local search realigns the candidate cursor to target tokens; every reused token is checked under the current target prefix, while repeated-token anchors can increase verification work.Unmatched positions are retired, allowing the target to continue alone.Source: §C.3 · p. 19

C.4 CORRECTNESS CONDITIONS AND PROOF

The proof states conditions under which target-decoupled proposals preserve greedy target outputs and state. The implementation and evaluation appendix describes the retained four-phone ASR cohort and its fixed runtime configuration.

C.4 CORRECTNESS CONDITIONS AND PROOF

  • With reference-equivalent verification and accepted-prefix state restoration, arbitrary proposals, budgets, and candidate recycling preserve target tokens and state, so completed outputs match the reference.The backend and protocol conditions must be validated for every admitted configuration.Source: §C.4 · pp. 19–20

D IMPLEMENTATION AND EVALUATION DETAILS

  • The four-phone ASR study retains all 688 full-audio test windows and 94,511 target tokens per phone, including capped outputs and throughput regressions.Native failures are retained with their execution boundaries.Source: D IMPLEMENTATION AND EVALUATION DETAILS · p. 20

D.1.1 DEVICES AND RUNTIME

  • The ASR runtime pairs a CPU Qwen3-ASR-Audio-0.6B source producer with a resident OpenCL 1.7B target, and source generation does not depend on newly committed target text.Each deployment uses its fixed drafter and preselected budget; Table 6 lists the comparison’s fixed models, placement, and proposal budgets.Source: §D.1.1 · pp. 20–21, Table 6 · p. 21

D.1.2 DATASETS, TASK COHORTS, AND TOKEN ACCOUNTING

This section describes the evaluation cohorts, decoding baselines, phone-specific calibration, and cost-based condition for throughput gains from target-decoupled overlap. It also reports a retrospective Redmi Note 9 Pro reanalysis and replay timing validation, whose scope is limited to phone token trajectories and context-matched timing profiles.

D.1.2 DATASETS, TASK COHORTS, AND TOKEN ACCOUNTING

  • The corpus contains 928 requests across ASR, two translation directions, and spoken QA; deduplication yields 913 audio inputs totaling 43,844.9815 seconds.The ASR cohort has 688 windows across five datasets, totaling 11.0 audio hours and 93,824 non-EOS target tokens; the combined 97,953-token count includes the first prefill-selected token and excludes EOS.Source: §D.1.2 · pp. 21–22
  • The Spoken SQuAD comparison uses a post-hoc length-conditioned subgroup of 80 questions and does not establish an unselected QA-wide gain.Questions were selected from 970 frozen items using a minimum of 16 useful target tokens, excluding EOS and the first prefill-selected token; the subgroup contains 1,895 useful tokens. Translation and QA TPS pools useful target tokens over post-prefill, resource-complete replay time.Source: §D.1.2 · p. 22

D.1.3 BASELINE IMPLEMENTATIONS

  • ASR Standard SD averages pooled-TPS results for K=5–8, while the evaluated SpecASR ASP port uses a 0.4 confidence threshold and at most 24 draft tokens.For translation and QA, SD and adapted SpecASR use real conditional proposals with budget 7 (physical K ≤8); their SpecASR runtime excludes the original masked-pair/TSP approach.Source: §D.1.3 · pp. 22–23

D.2.2 REDMI K70 PRO CALIBRATION

  • In one warmup plus five paired measured repetitions on a calibration audio, target-decoupled overlap reached 29.81 pooled TPS versus 15.41 TPS for serial budget 7, a 93.4% increase.Source: §D.2.2 · p. 24
  • On three K70 development windows, serial/TD throughput was 14.78/24.20 TPS on FLEURS, 9.62/15.80 on KeSpeech, and 11.85/12.52 on M4Singer.Replay signed relative TD-time errors were +0.6%, +2.0%, and −8.5%, respectively; two other windows yielded no measured pairs, but no corpus-test windows were removed.Source: §D.2.2 · p. 24

D.2.6 FIXED CONFIGURATIONS AND DEVELOPMENT SPLITS

  • Validation selected Redmi Note 9 Pro Qwen/budget 3 at 5.6233 TPS versus 5.3898 for budget 7; the fixed setting reached 5.8512 TPS on all 688 test windows.This is a retrospective reanalysis of preserved validation and phone profiles; test outcomes were not used in configuration ranking.Source: §D.2.6 · p. 26

D.2.7 OVERLAP, ADDED WORK, AND PERFORMANCE LIMITS

  • Overlap yields a strict throughput gain exactly when Oa > ∆D + ∆V + ∆H; added costs can be negative, and the bound may be unattainable.If ∆D + ∆V + ∆H ≥ min(Da, Va), no overlap can produce a strict speedup for fixed realized costs; raw acceptance alone does not determine throughput.Source: §D.2.7 · p. 27

D.3 REPLAY AGREEMENT WITH NATIVE TIMING

  • The K70 replay comparison uses the same phone-generated target and candidate sequences as native overlap, with post-prefill timing that includes producer completion.The passage defines signed TPS error and mean absolute error but does not report replay-agreement results.Source: §D.3 · p. 27
  • Validation used freshly calibrated, duration-matched audio, alternating run order, and median native AS2D latency; all completed windows matched the target reference and reached EOS.The initial check completed nine windows; a tenth stopped during calibration, after which five initial windows and five new windows were remeasured.Source: §D.3 · p. 27
  • Mean absolute TPS errors were 5.81% in the initial check and 6.54% in the fresh repeat.Table 7 summarizes the replay TPS errors for both checks.Source: Table 7 · p. 28, §D.3 · p. 28
  • In the earlier 12-window block-calibration comparison, mean absolute TPS error was 8.43% and pooled error was −9.39%.All 32 measured overlap outputs matched the target reference, but only 12 of 16 windows had complete, identical source sequences for timing comparison.Source: §D.3 · p. 28
  • These checks condition on phone token trajectories; context-matched profiles validate timing only, while headline corpus replay uses server-generated traces.The context-matched profiles do not replace the headline corpus profiles.Source: §D.3 · p. 28

D.4.1 FOUR-PHONE THROUGHPUT DISTRIBUTIONS AND COMPLETE RESULTS

The frozen deployment improves pooled throughput over target-only and Standard SD, while fixed-width controls show that fewer verification rounds do not necessarily yield higher complete decoding throughput. Across four phones, selected settings approach the hindsight oracle, though the reported oracle gaps are based on modeled cost sensitivities.

D.4.1 FOUR-PHONE THROUGHPUT DISTRIBUTIONS AND COMPLETE RESULTS

  • 14.68 pooled TPS improves throughput by 75.8% over target-only and 41.9% over Standard SD; per-dataset gains over the strongest baseline range from 17.0% to 52.0%.Table 8 compares phone-profile replay on the same 688 windows; the Note 9 Pro configuration was validation-selected and fixed for testing.Source: §D.4.1 · p. 28, Table 8 · p. 29

D.4.2 FIXED-WIDTH SD CONTROLS

  • On Xiaomi 11 Pro, Standard SD’s fixed-width mean is 10.34 TPS, with pooled rates ranging from 9.71 to 10.70 TPS across four widths.The mean is an equal-weight average of the four fixed configurations, not a random switching policy or validation-selected best setting.Source: §D.4.2 · p. 28, Table 9 · p. 29
  • On K70, fixed-width K8 outperforms K12 on all five datasets; K12 cuts verification rounds from 16,148 to 12,681 but raises draft forwards by 18.7%.K12 has lower complete SD throughput despite fewer target calls; the fixed-SD mean and ASP’s maximum of 24 draft tokens are distinct operating points, not a matched-length ablation.Source: §D.4.2 · pp. 29–30, Table 10 · p. 30

D.4.3 SELECTION HEADROOM AND FIXED CONTROLS

  • Selected/oracle pooled TPS was 27.92/28.54 on K70, 14.68/15.08 on Xiaomi 11 Pro, 28.40/29.03 on Xiaomi 14, and 5.85/5.96 on Redmi Note 9 Pro.The hindsight oracle selects a feasible source and fixed cap per window at zero selection cost; deployment settings do not use test outcomes.Source: §D.4.3 · p. 30
  • Across nominal, low-ratio, and high-ratio cost scenarios, pooled gaps span 1.94–2.19%, 2.00–2.88%, 2.17–2.24%, and 1.78–1.81% across the four phones.Concurrent costs above physical K8 are modeled; these are cost-model sensitivity ranges, not statistical confidence intervals or proof of native-corpus optimality.Source: §D.4.3 · pp. 30–31

D.4.4 TRANSLATION AND SPOKEN QA RESULTS

On paired QA requests, AS2D outperforms the strongest serial baseline, while acceptance and candidate coverage vary across translation and QA. Audits validate selected traces but do not establish corpus-wide native numerical equivalence.

D.4.4 TRANSLATION AND SPOKEN QA RESULTS

  • Conditional acceptance is 54.6% for English-to-Chinese, 63.8% for Chinese-to-English, and 84.7% for QA, but accepted proposals cover only 26.6%, 36.7%, and 28.3% of useful target output.The corresponding K > 1 verification-round rates are 26.8%, 33.7%, and 17.5%, so high acceptance among tested proposals does not ensure broad candidate coverage or large throughput gains.Source: §D.4.4 · p. 31
  • Audits reproduce selected records and checks, but do not establish corpus-wide native numerical equivalence.The QA audit matches 80 frozen audio and reference hashes and reproduces Target-only timing and token counts.Source: §D.4.4 · p. 31
  • AS2D reaches 9.617707 TPS on paired QA requests, 8.0% above SD and 23.7% above Target-only.AS2D uses budget 3, while the serial baselines use their reported budget 7.Source: §D.4.4 · p. 31

D.4.5 WORKLOAD AND NATIVE SCHEDULING CASES

Across workload cases, AS2D's gains depend on useful candidate supply, overlap, verification cost, and device conditions; physical overlap or acceptance alone does not predict net throughput. The section also defines the population and timing boundaries used to interpret these comparisons.

D.4.5 WORKLOAD AND NATIVE SCHEDULING CASES

  • All-window median TPS improvements over the faster serial method on Xiaomi 14 are 91.0% for LibriSpeech, 102.7% for FLEURS, 114.5% for KeSpeech, 100.4% for M4Singer, and 101.3% for AMI.Across four phones, 431 of 2,752 phone/window pairs improve TPS by at least 100% over both serial methods.Source: §D.4.5 · p. 31
  • On the shared 206-token read-speech input, AS2D reaches 40.42 TPS on K70 and 32.24 TPS on Xiaomi 14, while serial speculation helps K70 but slows Xiaomi 14.SpecASR drafting cost rises from 5.33 s on K70 to 9.29 s on Xiaomi 14.Source: §D.4.5 · p. 32
  • The 688-window slowdown percentages are window frequencies, not pooled speedups or quality regressions; the frozen deployment reduces slowdown frequency but does not guarantee every-window speedup.Every window receives equal weight, including capped outputs.Source: §D.4.5 · p. 32
  • Parakeet raises pooled TPS from 28.36 to 46.33 on Xiaomi 14 LibriSpeech despite a lower accepted/proposed ratio, because candidate supply is faster.Qwen drafting is physically overlapped for 97.4% of active time, yet 78.7% of verification rounds have physical K1 and no ready candidates.Source: §D.4.5 · p. 33
  • On AMI, Parakeet acceptance of 23.9% versus Qwen's 41.1% leaves K1 rounds at 84.5%, and pooled throughput rises only 2.9%, from 22.11 to 22.74 TPS.Verification and correction still determine useful progress when candidates rarely advance the target.Source: §D.4.5 · p. 33
  • On one K70 KeSpeech pair, target-decoupled drafting reaches 15.80 TPS versus 9.62 for serial drafting even as acceptance falls from 44.9% to 24.3%.On the contrasting M4Singer pair, it accepts none of 30 proposals and improves from 11.85 to 12.52 TPS (5.7%); these are individual observations, not dataset estimates.Source: §D.4.5 · p. 33
  • On K70 AMI, prefix-dependent lookahead hides 34.9% of drafting but only 6.2% is reusable after rejection and realignment; two native pairs gain 1.7%.This different runner illustrates why physical overlap alone does not establish useful speedup.Source: §D.4.5 · p. 34
  • On one AMI window, Parakeet/K = 4 reaches 16.25 TPS, 11.5% above target-only, while K = 8 accepts more additional tokens but falls to 15.66 TPS.Fewer target calls do not guarantee lower time when wider verification costs more; these repeated runs concern one window, not uncertainty across recordings.Source: §D.4.5 · pp. 34–35

D.5.1 CUMULATIVE REDMI K70 PRO ABLATIONS

The component removals show that target-decoupled overlap and budget configuration both contribute to the evaluated deployments, with gains varying by phone. A native on-demand request case also reports a peak throughput gain, while its sweep does not establish exact-output preservation.

D.5.1 CUMULATIVE REDMI K70 PRO ABLATIONS

  • Selecting budget 7 after screening limits 1–24 on 50 development windows raises serial TPS from 16.81 to 19.77.The deployment pairs this fixed budget with target-decoupled proposals and overlap, using an independent producer that reads no target text or target KV.Source: §D.5.1 · p. 35
  • Without budget configuration on Redmi K70 Pro, pooled throughput is 25.60 TPS, while the selected AS2D configuration is 9.0% faster.A sensitivity check for extrapolated large-width concurrent costs gives 25.42 pooled TPS; both are profile-replay estimates.Source: §D.5.1 · p. 36

D.5.2 XIAOMI 11 PRO COMPONENT REMOVALS

  • Under current Xiaomi costs, AS2D reaches 14.68 pooled TPS versus 11.00 without TD (a 33.4% increase) and 13.79 without BC (AS2D is 6.4% faster).Both component removals retain the same 688 test windows, Qwen source, device profiles, and post-prefill clock.Source: §D.5.2 · p. 36

D.5.3 XIAOMI 14 AND REDMI NOTE 9 PRO COMPONENT REMOVALS

  • AS2D reaches 28.40 TPS on Xiaomi 14 versus 27.84 without budget configuration and 14.55 without target-decoupled drafting; on Redmi Note 9 Pro it reaches 5.85 versus 5.07 and 3.84 TPS.The matched Redmi Note 9 Pro comparison shows a 52.5% gain from target-decoupled overlap.Source: §D.5.3 · p. 37
  • The w/o-BC Xiaomi 14 and Redmi Note 9 Pro controls use extrapolated concurrent costs for some widths above K8, with sensitivity scenarios yielding 27.81–27.98 and 5.06–5.07 TPS, respectively.These scenarios are cost-sensitivity checks, not confidence intervals or measured concurrent large-width phone executions.Source: §D.5.3 · p. 37

D.6.1 NATIVE ON-DEMAND REQUEST PROTOCOL AND BASELINES

  • At the 120s native on-demand checkpoint, AS2D's peak 78.0% TPS gain accompanies response time falling from 25.05s to 14.07s, with all four methods producing the same 173 body tokens.Across the 19-position sweep, AS2D meets the stated word-agreement bound at 17 positions; throughput does not establish exact-output preservation across the sweep or corpus-wide gains.Source: §D.6.1 · p. 37
  • On one 60s Xiaomi 14 window, AS2D reduces estimated request energy by 29.9% and energy per body token by 27.7%.The two formal responses contain the same 194 normalized words in the same order, but their raw token sequences differ.Source: §D.6.1 · p. 38
  • The energy estimates come from one window and one formal run per method, without independent external calibration or workload-level variability estimates.The case measures request energy under this deployment, not battery-life gains across workloads.Source: §D.6.1 · p. 38

D.6.2 SUSTAINED NATIVE STREAMING

In a paced native streaming case, cooling keeps the system on pace while the uncooled run accumulates backlog, but the deployment comparison does not isolate cooling. The related-work sections distinguish AS2D's response-generation focus and target-decoupled verification from other audio, asynchronous, mobile, and candidate-configuration approaches.

D.6.2 SUSTAINED NATIVE STREAMING

  • The native stream uses one 600s audiobook excerpt in 150 four-second blocks, with a resident 7B target, a 0.6B drafter, and budget three.Both cases retain 75% of target audio tokens, but source deadlines and starting thermal states differ, so the curves are descriptive deployment cases rather than an isolated cooling ablation.Source: §D.6.2 · p. 38
  • Over 600s, the cooled run averages 3.63s service per 4s block, while the uncooled run averages 4.68s and ends with 120.50s completion lag.The cooled run has p95 completion lag 4.68s; prefill is 61.6% of its complete service, and decoding acceleration alone does not guarantee sustained timing.Source: §D.6.2 · pp. 38–39
  • One run per condition and a fixed retention rate do not establish robustness across workloads or durations, and the study does not evaluate recognition accuracy under audio-token reduction.The paced-recording study measures native runtime behavior, not microphone/UI latency or an AS2D versus Target-only/SD comparison.Source: §D.6.2 · p. 40

E.1 ON-DEMAND AUDIO UNDERSTANDING

  • On-demand audio systems prepare or retrieve past-audio context, but complete replies still require decoding; AS2D targets phone request-to-response latency under a fixed audio-preparation protocol.Other systems use semantic memories, event records, streaming prefill, or selected KV states.Source: §E.1 · p. 40

E.2 SPEECH PROPOSALS AND TARGET VERIFICATION

  • AS2D adapts local draft reuse while letting an autoregressive audio-conditioned producer follow its own history; unlike some speech methods, it retains target verification and correction.Other approaches described include acoustic-alignment reuse, masked or multi-head search, and CTC confidence-gated verification.Source: §E.2 · p. 40

E.3 ASYNCHRONOUS AND MOBILE EXECUTION

  • Unlike asynchronous methods that advance work from verified or hypothesized target-text prefixes, AS2D draws candidates from audio and instruction, verifies ready candidates, or decodes alone.Rejection triggers candidate realignment while the producer continues; useful overlap depends on candidate readiness and measured concurrent costs.Source: §E.3 · p. 41

E.4 CANDIDATE STRUCTURE AND DEPLOYMENT CONFIGURATION

  • AS2D uses ordered candidate chains suited to causal-attention runtimes and configures a fixed task-specific drafter's budget from device profiles and development workloads.The target remains fixed, and configuration does not require test-time learning.Source: §E.4 · p. 41

F.1 TASK-DEPENDENT SPEEDUP AND COMPLEMENTARY SPECULATION

The discussion bounds AS2D’s gains by task alignment, interaction setting, device resources, and the cost of preparing audio before a request.

F.1 TASK-DEPENDENT SPEEDUP AND COMPLEMENTARY SPECULATION

  • Target-decoupled drafting may provide less useful candidate coverage for open-ended reasoning or dialogue that depends more on target decisions.Recognition, translation, and spoken-QA results do not establish comparable gains for those tasks.Source: §F.1 · p. 41

F.2 SUPPORTING FULL-DUPLEX DIALOGUE

  • The evaluated requests use fixed audio views and exclude user interruptions and multi-turn dialogue, so extending AS2D requires defining context updates and candidate eligibility.The prepared-cache sweep also does not establish incremental cache maintenance across successive requests.Source: §F.2 · p. 42

F.3 RESOURCE ASSUMPTIONS AND FIXED CONFIGURATIONS

  • Configurations remain fixed after selection, and the 1.80–2.67% gaps to the per-window oracle cover only a finite catalog under measured costs.The results establish neither global optimality nor adaptation to changing workload, thermal state, or competing applications.Source: §F.3 · p. 42

F.4 RESPONSE LATENCY AND PREPARATION COSTS

  • Decoding gains alone do not establish lower total energy or end-to-end latency when audio preparation is unfinished.The Xiaomi 14 energy measurement covers one prepared-cache request and excludes earlier audio preparation and external cooling.Source: §F.4 · p. 42

F.5 EVALUATION SCOPE

The evaluation combines phone-profile replay with limited native cases, and its output-equivalence claims are bounded by numerical variation and state-restoration requirements.

F.5 EVALUATION SCOPE

  • Corpus-wide comparisons use phone-profile replay, which excludes model loading and audio/prompt prefill and does not separately calibrate queue/cache-control costs or contention.ASR replay covers 688 full-audio test windows and 94,511 target tokens per phone on four separately calibrated profiles.Source: §F.5 · p. 42
  • Native cases cover two phones and separate protocols; they do not establish corpus-wide native gains, recognition quality under pruning, or stability beyond measured duration.The 10-minute continuous-input study's cooled run keeps pace with arriving audio, while the uncooled case develops backlog.Source: §F.5 · pp. 42–43

F.6 NUMERICAL SENSITIVITY ACROSS EXECUTION BACKENDS

  • Finite-precision execution can change outputs across backends or verification shapes, even with greedy decoding, because rounding and reduction schedules can alter nearly tied token rankings.The cited discussion distinguishes repeatability at a fixed configuration from batch invariance.Source: §F.6 · p. 43
  • Native checks retain output discrepancies, including five of 40 Note 9 Pro resident-source calibration episodes that fail exact output matching.The observations do not identify each discrepancy's cause and preclude exact-output claims for affected runs.Source: §F.6 · p. 43
  • Replay cannot validate partial-KV repair; state-equivalence claims require validated accepted-prefix restoration with repair costs included.Proposition 1 is conditional on reference-equivalent execution and state restoration, not bitwise reproducibility across implementations.Source: §F.6 · p. 43
Loading 2609.37617v1…