PaperlayerView exact revision on arXiv

Source-linked AI summary

Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readiness

Dan Ben-Ami, Kobi Cohen, Chaim Baskin

arXiv:2610.08560v1cs.CVcs.CLcs.LG

TL;DR

Streaming video-language models must decide whether evidence for a question has arrived, not just what to answer. This paper tests whether frozen models already encode that readiness, and uses a linear readout to time answers. Readiness is readable across models and question conditions, and Readiness Gating improves accuracy by up to 9.75 pp at matched video duration.

  • Problem

    Streaming systems must decide whether evidence for the current question has arrived, while existing approaches learn a separate trigger and it is unknown whether frozen VideoLLMs already encode readiness.

    Source: §1 · p. 1

  • Method

    The paper labels readiness from timestamped evidence independently of model answers, probes frozen residual-stream representations, and uses the readout in an answer-timing policy.

    Source: §1 · pp. 1–2, §3.2 · p. 4

  • Results

    Readiness decodes in all seven tested models, and Readiness Gating improves accuracy by up to 9.75 pp at matched video duration.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Frozen VideoLLMs encode a linearly readable, question-conditioned evidence-readiness signal that can be used to time answers.

    Source: §1 · pp. 1–2

    Long-stream latency and memory growth remain outside scope, and gated stopping needs an operating point that can fail under label shift.

    Source: §5 · p. 10

Abstract

from arXiv · show

Streaming video-language models must decide not only what to answer, but whether the evidence needed for the current question has arrived. Existing systems learn that decision as a separate trigger; we ask whether an unmodified model already computes it. We show that frozen VideoLLMs carry a linearly readable evidence-readiness signal, labelled from timestamped evidence rather than from model output. It decodes in all seven models of a shared byte-identical evaluation (AUROC 0.733-0.905 under the strictest not-ready sampling, where a fitted clock is near chance), and a probe fitted without any of a benchmark family's footage still reads that family. It is question-conditioned: on byte-identical windows, changing only the question reverses the readout on 66.1% of pairs, while every question-blind control is at chance by construction. The model can answer incorrectly and still encode readiness: AUROC remains 0.722 among wrong answers. Readiness also beats uncertainty estimators and their supervised combination on latency-matched answer selection, and tracks independent human judgments more closely than confidence. Released streaming triggers are also linear readouts, yet a trained trigger read on its own base model's activations is approximately orthogonal to readiness and decodes it far less accurately than a probe. We turn the readout into Readiness Gating, an answer-timing policy that improves accuracy by up to +9.75 pp at matched video duration with negligible computational overhead. How much it gains varies with the accuracy headroom the task makes available: across 26 configurations the gain tracks that headroom, and an intervention that moves it over identical pixels moves the gain with it.

1 INTRODUCTION

Streaming video-language models must decide both what to answer and whether the evidence has arrived, a question existing systems address with learned triggers. This paper finds that frozen models already encode question-conditioned readiness, which can be read out and used to time answers.

1 INTRODUCTION

  • Streaming changes the task from answering with a complete clip to deciding whether enough evidence has arrived, while existing systems learn a separate readiness mechanism.The paper asks whether that state is already present in a frozen VideoLLM.Source: §1 · p. 1
  • Evidence availability is distinct from confidence and correctness, and the paper labels it using question-specific evidence timestamps fixed before model outputs are seen.A model may answer confidently from prior knowledge before evidence arrives, or fail despite having seen the relevant pixels.Source: §1 · p. 1, Figure 1 · p. 2
  • Readiness is linearly readable in the frozen residual stream after the first transformer block, despite not being a next-token training target.The training objective supervises what to say, not whether delivered frames settle the question.Source: §1 · p. 1
  • Changing only the question reverses the readiness readout on byte-identical pixels, linking the signal to vision–language interaction.The signal also remains readable among wrong answers and across seven models and held-out benchmark footage.Source: §1 · p. 2
  • Readiness Gating uses the readout to time answers, improving accuracy by up to 9.75 pp at matched video duration without an additional forward pass.Its gains vary with task headroom.Source: §1 · p. 2

2 RELATED WORK

The related work spans streaming answer timing, temporal evidence and frame selection, uncertainty and answerability, and representation probing and intervention. This paper positions readiness as an internal, question-specific visual-evidence signal in frozen models.

Streaming video and answer timing

  • Streaming VideoLLMs use training, memory, proactive-response modules, or learned triggers to decide when to answer; this work asks whether frozen models already contain a reusable readiness state.Released triggers that are linear readouts can be directly compared with the paper’s probe.Source: §2 · p. 2

Temporal grounding and evidence

  • Temporal grounding and adaptive frame selection localize or select video evidence, while this work uses evidence annotations to test whether delivered frames support a question.An annotated span may not be the earliest or only sufficient evidence, bounding the interpretation of the measure.Source: §2 · p. 3

Uncertainty, abstention, and answerability

  • Calibration, consistency, entropy, and abstention methods assess output correctness, stability, or answerability; readiness instead targets question-specific visual evidence in observed frames.The paper evaluates readiness separately at representation and decision levels.Source: §2 · p. 3

Probing and activation intervention

  • Probing can expose represented information that generation does not express, while activation interventions test behavioral effects whose interpretation depends on intervention and control.The paper pairs decoding with controls and treats sufficiency and necessity as separate hypotheses.Source: §2 · p. 3

3 MEASURING EVIDENCE READINESS

The section defines timestamp-based readiness labels across video formats, then tests frozen residual-stream representations with probes and controls. It also specifies Readiness Gating as a policy for using the readout to time or select answers.

3.1 VIDEO INPUTS AND READINESS LABELS

  • The evaluation uses timestamped benchmarks and additional video formats; human ratings assess whether evidence is answerable, while both-sided not-ready sampling reduces the time-only baseline from 0.950 to 0.527 AUROC.The paper notes that formats have different labels and their AUROCs should not be compared across formats.Source: §3.1 · pp. 3–4
  • For fixed windows, readiness requires full coverage of the evidence hull; for growing prefixes, the binary label marks evidence onset, and annotation-selected checkpoints do not establish absolute deployment latency.Partial-coverage cases are excluded from binary probe fitting but retained as not ready for policy evaluation.Source: §3.1 · pp. 3–4

3.2 FROZEN REPRESENTATIONS, PROBE, AND CONTROLS

  • A nested-cross-validated logistic-regression probe reads frozen residual-stream states, with whole videos held out and layer-0 visual states treated as a measured baseline.The probe tests linear availability rather than teaching the model readiness.Source: §3.2 · p. 4
  • The null and competing readouts include within-group shuffles, elapsed time, confidence, pixel and motion statistics, and eleven uncertainty estimators adapted from five method families.Elapsed time’s informativeness depends on the evaluation pool and sampling geometry.Source: §3.2 · p. 4
  • Crossed quads pair two questions with disjoint evidence hulls and two windows, so each window is ready for one question but not the other; the DiD cancels fixed question and window effects.Question-blind and pixel-only readouts receive zero DiD and AUROC 0.5 by construction.Source: §3.2 · pp. 4–5
  • Across Table 1’s nine tests, readiness exceeds all three comparison columns on every row by 0.133 to 0.220.Rows differ in geometry and strictness, so AUROCs should be compared within rows, not across them.Source: Table 1 · p. 5

3.3 READINESS GATING

  • Readiness Gating uses readiness and confidence thresholds to stop at a checkpoint, or selects the cached answer with highest readiness at a fixed deadline.Both modes add a 1.06 µs dot product to a 251 ms forward pass; reported accuracy differences compare with confidence alone at matched video duration.Source: §3.3 · p. 5

4 EVIDENCE READINESS: PRESENCE, SEPARATION, GENERALITY, UTILITY

Frozen VideoLLMs encode question-conditioned evidence readiness that remains readable across models, footage, and settings, even when answers are wrong. Readiness can guide answer timing, but its gains depend on task headroom, and released triggers do not necessarily read the same signal.

4.1 A COMPUTED, QUESTION-CONDITIONED READINESS SIGNAL

  • Readiness decodes at AUROC 0.985 on recolored cue-object windows and tracks timestamp labels about as well as human raters, whose judgments also predict answerability.The cue-object evaluation holds out cue objects and separates the queried cue from an equally salient irrelevant one; raters and probes recover timestamp labels at AUROC 0.681–0.778 and 0.671–0.791, respectively.Source: §4.1 · pp. 5–6
  • On natural NExT-GQA pairs, changing only the question reverses readiness on 66.1% of swaps, while every window-only statistic is at chance by construction.The predicted direction of question-conditioned readout changes corresponds to a +14.0 pp model-accuracy shift; readiness also rises with evidence seen relative to total evidence segments.Source: §4.1 · p. 6

4.2 SEPARATION FROM CONFIDENCE, CORRECTNESS, AND OVERT BEHAVIOUR

  • Within fixed confidence bins, readiness remains decodable, and all eleven tested uncertainty estimators lose to readiness on both tested answer-selection settings.Adding a supervised combination to gating gives no detectable gain; the table reports paired margins of +7.1 and +10.7 pp over confidence.Source: §4.2 · p. 6, Table 2 · p. 7
  • Among HD-EPIC wrong answers, readiness AUROC is 0.722, and it remains 0.970 among HERBench present-but-wrong examples.These results separate evidence availability from whether the model correctly uses that evidence.Source: §4.2 · pp. 6–7

4.3 GENERALITY ACROSS MODELS, FOOTAGE, AND TASKS

  • Across seven models, readiness AUROC is 0.733–0.905 under the primary not-ready sampling, and each language-position readout exceeds its visual layer-0 baseline.All seven peak between 0.5 and 0.8 of decoder depth; model answer probabilities rank the rows at only 0.514–0.651.Source: §4.3 · p. 7, Figure 2 · p. 8
  • On five held-out benchmark families, transferred probes reach 0.653–0.761 AUROC, averaging 0.922 of family-fitted ceilings, but the operating point needs target calibration.On OVO, ranking transfers at 0.748 AUROC, while the carried threshold yields 13.8% accuracy; recalibrating it on eight target videos recovers 86.5%.Source: §4.3 · p. 8

4.4 READINESS GATING AND THE ATTAINABLE-BAND RELATIONSHIP

  • At matched latency, Readiness Gating gains +9.75 pp on HERBench shot identification, while gains across 26 configurations track attainable accuracy headroom.A distractor intervention over identical pixels moves both end-of-video accuracy and headroom, with the gating gain falling from +4.66 to +1.10 pp.Source: §4.4 · pp. 8–9

4.5 RELEASED STREAMING TRIGGERS VERSUS THE READINESS PROBE

  • On MMDuet’s own activations, the readiness probe reaches AUROC 0.843 versus 0.607 for its trained relevance head, whose direction is approximately orthogonal to readiness.The comparison uses the same captured activations; the two directions have absolute cosine 0.016 against a random-direction null with 99th percentile 0.043.Source: §4.5 · pp. 9–10

5 DISCUSSION

Frozen VideoLLMs contain a linearly readable signal for whether the current question’s evidence has arrived, and Readiness Gating uses it for answer timing. The discussion also bounds the evidence: several aspects of deployment and causal scope remain outside the evaluations.

5 DISCUSSION

  • The paper concludes that frozen VideoLLMs contain a linearly readable, confidence-separable readiness state that is portable in ranking.Readiness Gating turns the readout into an answer-timing policy whose gain varies with task headroom.Source: §5 · p. 10
  • Released streaming triggers transfer poorly to the readiness labels; the tested trigger is approximately orthogonal to readiness on its own base model’s activations.A probe decodes readiness far more accurately on those same activations.Source: §5 · p. 10
  • Long-stream latency and memory growth remain outside scope, and stopping requires an operating point that can fail under label shift.The causal evidence covers one primary intervention site and two video-language models, establishing sufficiency there rather than necessity.Source: §5 · p. 10

AI USE STATEMENT

Generative AI tools supported writing, literature discovery, and specified experimental procedures. Authors state that they checked the resulting text and references and take responsibility for the work.

AI USE STATEMENT

  • AI tools helped draft and revise text from author-supplied content, improve readability, organize the manuscript, and format references, tables, and captions.Authors report reading, checking, and editing each such passage.Source: AI USE STATEMENT · p. 10
  • AI tools helped identify candidate papers and relevant passages, while authors read the cited papers and checked claims against primary sources.No citation was included solely on a tool’s assertion.Source: AI USE STATEMENT · p. 10
  • AI was also used to construct and screen benchmark items and as an evaluation baseline; the authors take responsibility for the text, data, analyses, and results.The statement points to relevant appendices for documentation.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The study analyzes publicly released models and video question-answering benchmarks without recording new video. Its human-rating procedure and handling of identifiable footage are described alongside privacy considerations.

ETHICS STATEMENT

  • The work analyzes frozen, publicly released video-language models and publicly released benchmarks, and records no new video.The listed benchmarks include HD-EPIC, NExT-GQA, ST-Evidence-MCQ, ST-Evidence-Gen, ActivityNet-Captions, Oops!, HERBench, and OVO-Bench.Source: ETHICS STATEMENT · p. 11
  • Three unaffiliated institutional raters judged 900 screens of public footage, volunteered without compensation, and could stop at any point.The study collected no personal or demographic data and releases no participant-level data.Source: ETHICS STATEMENT · p. 11
  • The authors note that applying an evidence-arrival signal to camera streams of people raises surveillance and privacy concerns separate from the accuracy questions studied.Identifiable HD-EPIC footage is used under its release terms, with no derived media released.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

The reproducibility statement describes frozen-model evaluation and the fitted probe, while the appendix guide maps research questions to supporting evidence. The geometry discussion distinguishes readiness labels and streaming input regimes, including their timing confounds.

REPRODUCIBILITY STATEMENT

  • Models remain frozen throughout; the only fitted object is a linear probe on frozen activations, with generalization evaluated on whole-video held-out groups.The readout layer and regularization are selected using an inner held-out-group loop, and confidence intervals use cluster bootstrapping over groups.Source: REPRODUCIBILITY STATEMENT · p. 11

APPENDIX GUIDE

  • The appendix guide directs readers to evidence on readiness labels, signal measurement, question specificity, generalization, answer timing, additional tests, reproducibility, and scope limits.It maps these questions to Appendices A–M.Source: APPENDIX GUIDE · p. 18

A STREAMING GEOMETRIES AND READINESS LABELS

  • Readiness labels measure coverage of annotated question evidence in delivered frames before model output is inspected, distinct from perceived sufficiency and answer correctness.For fixed-span windows, readiness concerns the current input and can return to not ready when evidence leaves the window.Source: A STREAMING GEOMETRIES AND READINESS… · p. 18

A.1 THE GEOMETRIES

  • Growing prefixes make elapsed time informative, while fixed-span evidence-agnostic policy ladders yield clock AUROC of 0.500–0.506; other fixed-span pools can retain timing shortcuts through negative sampling.Prefix endpoints placed at evidence onset and offset make checkpoint index informative; the reference containment window combines constant span with an evidence-agnostic grid.Source: §A.1 · p. 18

A.2 WHERE THE COVERAGE THRESHOLD SITS

This section examines how the operational readiness threshold and negative-sampling choices affect the readout and the policy’s attainable benefit. It also checks graded evidence, independent human judgments, and where readiness appears in the model’s activations.

A.2 WHERE THE COVERAGE THRESHOLD SITS

  • Under containment, prefix-readout AUROC is 0.7224 on NExT-GQA and 0.7292 on ST-Evidence, versus 0.733 and 0.722 at onset; treating partial rows as not ready gives 0.656 and 0.637.The containment-versus-onset change is at most 0.011, while including partial rows lowers the readings because those checkpoints already show some evidence.Source: §A.2 · pp. 18–19, Table A.2 · p. 19

A.5 GRADED EVIDENCE

  • On fractional hull coverage, readiness scores correlate within items at Spearman ρ = 0.454, 0.490, and 0.558 on NExT-GQA, ST-Evidence, and ST-Evidence-Gen; raw segment count can instead produce a pooled ρ = 0.133.The fractional target is seen segments divided by K; a layer-0 readout and within-video shuffle remain near zero on the fractional-coverage analysis.Source: §A.2 · p. 19, §A.5 · p. 20

A.2 WHERE THE COVERAGE THRESHOLD SITS

  • On the reference 16 s window, changing the label from onset to containment moves the evidence-oracle advantage from +2.1 to +9.0 pp and the attainable band from 3.7 to 10.5 pp.The band depends on the label convention and is a reference for policy gain, not a guaranteed upper bound.Source: §A.2 · pp. 19–20

A.3 WHERE THE NOT-READY ROWS COME FROM

  • With both-sided negatives, the fitted clock falls to 0.5154 and 0.5271, while the probe-to-clock AUROC-drop ratio ranges from 0.0864 to 0.2286 across seven models.Before-only sampling lets the clock reach 0.9403 and 0.9497; the both-sided regime is reported as primary.Source: §A.3 · p. 20, clock tracking = · p. 20

B.1 INSTRUMENT

  • Two primary raters agreed on the three-category readiness verdict for 83.0% of 900 screens, with pooled Cohen’s κ = 0.717.A third rater judged only the 317 screens where the first two disagreed on either question.Source: §B.1 · p. 21

B.2 THE PROBE READS THE ANNOTATION AT RATER LEVEL

  • Table B.1 reports overlapping intervals for raters and the probe on timestamp labels, but this human check covers three evaluations, not controlled composites, model-axis, or transfer results.The raters’ own verdicts are also a distinct readiness target from timestamp coverage.Source: Table B.1 · p. 22, §B.3 · p. 22, §B.6 · p. 23

B.4 ASKING A MODEL DIRECTLY

  • On the tested screens, the external judge’s agreement with people remains low after removing its yes-bias, while raters’ answer accuracy rises in evidence-containing windows and can fall when evidence leaves a fixed-span window.The external judge is one zero-shot model in one configuration with a small reasoning budget.Source: §B.4 · p. 22, §B.5 · p. 23

C.1 WHERE WE READ

  • Readiness is at chance at layer 0 at language positions, rises with depth, and is carried by the language stream rather than the pooled visual representation.The primary readout position is the final prompt token; cross-model comparisons use matched depth fractions.Source: §C.1 · p. 23

C.2 PROBE AND EVALUATION

The evaluation fits linear probes to frozen-model activations and uses held-out groups, measured baselines, and a crossed-question design to test what readiness information is available and whether it depends on the question.

C.2 PROBE AND EVALUATION

  • An L2-regularized logistic regression predicts objective readiness from standardized frozen-model activations, so success indicates linear decodability rather than readiness learned by changing model weights.The primary state is the residual-stream activation at the final prompt token before answer generation.Source: §C.2 · p. 24
  • The evaluation holds out whole video groups or cue objects, nests layer and regularization selection inside training folds, and bootstraps confidence intervals over groups.The paper reports held-out AUROC, with AUPRC for imbalanced classes; graded targets use ridge regression and additional ranking and error metrics.Source: §C.2 · p. 24

C.3 THE CONTROL BATTERY

  • The final-language-position layer-0 control is exactly 0.500, while pooled-vision layer 0 reaches 0.579–0.683 under clock-balanced sampling and is a measured baseline, not a null.The paper also measures shuffled-label nulls, timing, confidence, pixel-statistics, and evidence-count baselines rather than assuming they are at chance.Source: §C.3 · p. 25

C.4 THE CROSSED-QUAD GUARANTEE

  • The crossed-quad design pairs two questions with two windows, each containing one question’s evidence, and reports held-out AUROC, flip rate, difference-in-differences, and contained-question accuracy.Both orderings are required to avoid a systematic position skew; the model’s accuracy difference is +0.1402 [0.1159, 0.1634].Source: §C.4 · p. 26
  • Because each window appears once as ready and once as not ready, video-only statistics receive identical inputs across labels and are forced to AUROC 0.500 in the crossed design.The guarantee depends on both members of each pair being assigned to the same fold; it does not hold for the controlled composites’ held-out-object protocol.Source: §C.4 · p. 26

D DATASETS, POOLS, MODELS, AND PROMPTS

This appendix documents the source and construction of the evaluation pools, their footage overlap, model and prompt configurations, and quality controls. It describes evidence-based readiness labels and pool-specific annotations, filters, readiness rates, and baselines that shape how results are interpreted.

D.1 BENCHMARK FAMILIES AND EVALUATION POOLS

  • Table D.1 distinguishes source and builder pools from the rows actually fitted or scored, which filtering and negative sampling can subset.The table’s source-video counts are not necessarily the evaluation unit; representation results use the geometry, label, and scored-row details reported separately.Source: §D.1 · p. 27, Table D.1 · p. 27
  • NExT-GQA and ST-Evidence provide human-grounded evidence for everyday-video questions, with readiness based on the delivered fraction of annotated evidence (Xiao et al., 2021; Xiao et al., 2024; Wang et al., 2026).NExT-GQA evidence can comprise disjoint segments; ST-Evidence uses temporal segments from its multiple-choice task and contributes proportionally more multi-segment items to the crossed-quad pool.Source: §D.1 · pp. 28–29
  • The generation companion retains four sources after lexical and duration filters, with per-source readiness rates of 0.573–0.903.The filters remove 376 position-leaking questions and a further 686 items below 10.0 seconds; pool positive rates vary across designs, including 0.500 on matched-pair and crossed designs and 0.13–0.19 on short-window substrates and the external benchmark.Source: §D.1 · pp. 28–29
  • On 87 movie trailers, shot ordering uses four evidence windows per item, while shot identification asks which single described shot has been seen.Ordering evidence spans nearly the full trailer, making delivered fraction and revealed-shot count similar; identification has one ready checkpoint per item.Source: §D.1 · p. 29
  • ActivityNet-Captions items use caption spans as evidence, while Oops! uses annotated failure onsets as point evidence (Krishna et al., 2017; Epstein et al., 2020).ActivityNet-Captions varies distractor difficulty over identical pixels; Oops! onsets have a median time of 3.50 s and fractional position of 0.45, and retained items limit annotator disagreement to 0.5 s.Source: §D.1 · p. 29
  • OVO-Bench serves only as a zero-shot transfer target, and its Yes/No format yields non-chance trivial baselines.On step rungs, always answering Yes scores 0.6689 versus the model’s 0.6290; on the evidence-agnostic clock ladder, always answering No scores 0.8694 versus the model’s 0.8025.Source: §D.1 · p. 29

D.2 CONSTRUCTED ITEMS: WHAT WE AUTHORED, AND HOW

  • Across four constructed evaluations, questions, answers, and distractors are authored from source annotations, while readiness labels always come from source timestamps.ActivityNet distractors receive lexical screening only (text-only scores 0.155 and 0.147 versus 0.200 chance); Oops! answers are model-mediated and its options undergo verification screens.Source: §D.2 · pp. 29–30, Table D.2 · p. 30

D.3 FOOTAGE INDEPENDENCE

  • The two NExT-QA evaluations share 221 videos and 240 item identifiers, so they replicate across annotations rather than independent video; HD-EPIC and HERBench also reuse footage.The paper treats families, not nominal evaluation pools, as the unit of footage independence.Source: §D.3 · p. 31

D.4 MODELS

  • Model families are defined by language backbone, and the short-window model-axis varies Qwen3 (2B, 8B, and 32B), GLM-4, Llama-3.1, and Mistral-Nemo on one manifest.The table lists evaluated models and probes layers by fractions of each model’s depth; systems sharing a backbone are not separate families for this purpose.Source: §D.4 · p. 32, Table D.3 · p. 32

D.5 FRAME BUDGETS AND PIXEL QUALITY CONTROL

  • Frame counts stay constant within each geometry, while trailer pixel filtering is excluded from the answer-timing ladder so it cannot remove policy choices.The trailer filter drops 18.8% of rows and removes the only reachable stop for 137 of 956 items; probing uses pixel-cleaned rows instead.Source: §D.5 · p. 33, Table D.4 · p. 33

D.6 PROMPTS

  • Prompts use fixed-count multiple choice, with deferral added as a configuration-level option that makes commitment observable for behavioral comparisons and interventions.The same grounded-prefix rows score 0.729 with five options and 0.695 with six; option counts and chance rates vary across benchmarks.Source: §D.6 · pp. 33–34

D.7 LICENSING, DATA AVAILABILITY, AND HUMAN SUBJECTS

  • Readiness labels use benchmark timestamps, while human input is limited to a detector audit and three unaffiliated raters’ 900-screen check; benchmark licences vary materially.No personal or demographic data were collected, and no video frames are redistributed.Source: §D.7 · p. 34

E.1 DECODABILITY, EVALUATION BY EVALUATION

Frozen VideoLLMs encode evidence availability in a question-conditioned signal that remains readable across models, answer correctness, and evaluation controls. Its persistence when time is uninformative, and its advantage over confidence-based alternatives, support treating readiness as distinct from confidence or correctness.

E.1 DECODABILITY, EVALUATION BY EVALUATION

  • 16 of 17 controlled-composite objects read above 0.98 under held-out-object cross-validation.Source: §E.1 · p. 35

E.2 QUESTION-CONDITIONING, AT FULL PRECISION

  • On byte-identical natural-video windows, changing only the question reverses the readout on 0.6614 of NExT-GQA swaps and 0.7074 of ST-Evidence-MCQ swaps.The crossed design forces question-blind visual controls to 0.500 by construction; controlled composites also show a 0.9808 reversal rate across 520 pairs.Source: §E.2 · pp. 35–36

E.3 CONFIDENCE AND CORRECTNESS

  • Readiness AUROC is 0.7224 among wrong answers and 0.7534 among correct ones, with availability contrasts larger than correctness contrasts within each group.The readout also remains informative within confidence bins, and residualizing it on confidence changes nothing; the paper cautions that these are prediction-level, not geometric, separation tests.Source: §E.3 · p. 37

E.4 UNCERTAINTY ESTIMATORS, ONE ROW AT A TIME

  • Every uncertainty alternative loses to readiness on latency-matched answer selection, and adding a supervised head to the probe yields no detectable gain.The comparison is evaluated on Oops! and ActivityNet-Captions; even the strongest single baseline loses selection on both substrates.Source: Table E.4 · p. 38, §E.4 · p. 39

E.5 GRADED EVIDENCE

  • 0.827 readiness AUROC on graded prefixes exceeds a within-video shuffle of 0.578, while score tracks the fraction of evidence segments seen after controlling for video progress.Re-asking those prefixes with an unrelated video’s question reduces AUROC from 0.832 to 0.546 and sharply weakens the score’s association with evidence seen and video progress.Source: §E.5 · pp. 39–40

F.1 THE FULL MODEL AXIS, ALL FOUR NEGATIVE-SAMPLING REGIMES

  • 0.7329–0.9046 readiness AUROC across seven models under both-sided sampling, while the fitted elapsed-time clock is 0.5271.The clock reaches 0.9497 under before-only sampling, where it exceeds every probe; that regime is therefore not strong evidence of readiness.Source: §F.1 · p. 40, Table F.1 · p. 40

F.2 WHERE IN DEPTH THE SIGNAL APPEARS

  • Readiness decodability rises with depth on two natural-footage replications, from near-chance AUROC at the shallowest sampled site to higher AUROC at the deepest.Across the full grid, the language-position readout exceeds the visual layer-0 baseline in all 28 cells; the fixed final-language-token control is 0.500 by construction.Source: §F.2 · p. 41

F.3 HOLDING OUT A BENCHMARK FAMILY

Readiness transfers across benchmark footage and can be read across model bases, but the evidence bounds what those transfers establish. A controlled cross-attribute test also shows transfer between non-aligned readiness readouts, while external transfer preserves ranking rather than calibration.

F.3 HOLDING OUT A BENCHMARK FAMILY

  • A probe trained without a benchmark family’s footage transfers to that family, with visual, shuffle, and elapsed-time controls near chance across held-out cells.The NExT-GQA and ST-Evidence-MCQ footage is held out together because the benchmarks share 221 videos; each transfer cell also lies inside the source union’s frame-density envelope.Source: §F.3 · p. 41, Table F.2 · p. 42

F.4 READING ONE MODEL’S PROBE IN ANOTHER MODEL’S BASIS

  • 0.7134 median aligned-transfer AUROC at rank 128 exceeds the same-rank random-rotation null of 0.4963 across 42 ordered model pairs.The aligned readout retains a median 0.8983 of each target model’s fitted ceiling, but the mapping preserves generic shared structure, so the supported claim is that readiness is not model-idiosyncratic—not that it is the shared direction.Source: §F.4 · p. 43, Table F.3 · p. 43

F.5 TRANSFER ACROSS THE ATTRIBUTE THE READINESS IS ABOUT

  • A probe trained on colour questions transfers to size questions at AUROC 0.9380, and the reverse transfers at 0.9954, despite a cosine of 0.2656 between the readiness directions.This is an existence result on controlled composites from one source and one model, not a claim about natural footage.Source: §F.5 · p. 43

G EXTERNAL TRANSFER AND THE OPERATING POINT

G.1 CARRYING A FROZEN PROBE TO AN EXTERNAL BENCHMARK

A frozen readiness probe transfers its ranking to an external benchmark, but its threshold can fail when prevalence shifts. Readiness Gating evaluates answer timing at matched latency, and its measured gains vary with task headroom and which items can actually be changed by stopping earlier.

G.1 CARRYING A FROZEN PROBE TO AN EXTERNAL BENCHMARK

  • 0.7484 frozen-probe AUROC on OVO-Bench is below the target-fitted probe’s 0.8322, while the within-video shuffle is 0.5085 and the elapsed-time clock is 0.4511.On the same target rows, model confidence and other obvious proxies rank below chance; the frozen probe is applied without target fitting or rescaling.Source: §G.1 · p. 44

G.2 REPAIRING THE OPERATING POINT, AND WHAT THE REPAIR IS WORTH

  • 0.8079 expected calibration error accompanies the carried threshold on OVO-Bench, where 99.28% of rows are called ready despite a 0.1306 positive rate.Refitting only the threshold on eight labelled videos closes 92.7% of the mean threshold gap in base-rate-shifted cells, but the same labels worsen accuracy in balanced pools.Source: §G.1 · p. 44, §G.2 · p. 44

H.1 THE UNIT IS A STOP DECISION

  • Gated stopping combines readiness and confidence ranks, whereas fixed-deadline selection chooses the cached answer from the highest-readiness window; their gains are different statistics.The stopping gain compares against confidence alone at equal latency; selection compares readiness against a rival monitor at the same deadline. Reported stopping gains match latency in seconds, which can differ from rung matching.Source: §H.1 · p. 46, §H.2 · p. 46

H.3 GUARDS

  • Every timing result is checked for matched latency, a fixed-time policy that does not consult the model, and whether distinct-frame counts track the label.A result that fails the anchoring check is not reported as a timing result.Source: §H.3 · p. 46

H.4 THE ATTAINABLE BAND

  • The attainable band is the accuracy difference between the evidence oracle and answering at the video’s end, not a guaranteed upper bound.Removing annotated evidence costs only 7.6, 8.4, and 5.2 pp on three benchmarks, and changing the readiness threshold from onset to containment changes the oracle’s ready windows.Source: §H.4 · p. 46

H.5 A CONTROLLED CHANGE IN HEADROOM

  • 34.5 pp attainable band and +4.66 pp stopping gain with same-action-class distractors contrast with 17.5 pp and +1.10 pp with uniform distractors, with visual evidence held fixed.This changes answer format rather than intervening on headroom alone; across 26 configurations the fitted slope is +0.148 pp gain per band pp, and the band tracks but does not determine gain.Source: Figure H.1 · p. 47, §H.5 · p. 47

H.6 DECIDABLE ITEMS AND FLAT LADDERS

  • Under the controlled distractor change, the flat-item share rises from 25.7% to 59.6%; flat ladders cannot be changed by a timing policy.Oracle comparisons also require naming the tie-breaking convention because multiple ready windows can be equally good stops.Source: §H.6 · p. 47, §H.7 · p. 47

H.8 DECISION-RULE LEVERS

  • Memory amplifies an existing thresholded-policy effect and replicates out of sample at +0.1087; it does not create an effect or substitute for attainable band.Source: §H.8 · p. 48

H.9 THE BAND REGRESSION

Across 26 configurations, the observed relationship between accuracy headroom and gating gain is strong but not a universal conversion rule. The section also sets out the scope and asymmetry of comparisons with published streaming triggers.

H.9 THE BAND REGRESSION

  • pp = 0.1482 bandpp −0.10, with r = 0.9399 and r2 = 0.883, summarizes the observed configurations rather than establishing a universal conversion rate.The controlled contrast and pooled fit agree in sign and scale.Source: §H.9 · p. 48
  • On Oops!, the gate converts 0.48 of a 12.5 pp band where the fit predicts 0.15, showing similar headroom can yield several-fold differences in extracted gain.The authors caution that a large band is not a promise of a large gain.Source: §H.9 · p. 48
  • Across 26 configurations, Spearman’s ρ is 0.793 [0.550, 0.926], and across the 23 remaining configurations it is 0.700 [0.383, 0.874]; rank order is more stable than the least-squares relationship.Three sparse four-option-burst configurations have bands above 0.6 and carry the least-squares fit; without them, the slope is +0.1139 at r2 = 0.479.Source: §H.9 · p. 48

I COMPARISON WITH PUBLISHED STREAMING TRIGGERS

  • Released triggers are linear residual-stream readouts, but the comparison is asymmetric: rivals are zero-shot while the probe is fitted on the substrates and scored out of fold.StreamReady and StreamOV had released no code, and StreamBridge withheld its activation-model weights; none could be driven end-to-end.Source: §I.1 · pp. 48–49

I.1 THE TRIGGERS, AND HOW THEY WERE DRIVEN

  • MMDuet ships as a 0.13 GB adapter at 2 fps, while Dispider decides once per 16 s; VideoLLM-online was obtained but not run.The rivals were driven in their own environment and framework pins, with no refitting or new footage.Source: Table I.1 · p. 49, §I.1 · p. 49

I.2 VALIDATION: THE SAME HEAD ON ITS OWN BENCHMARK

The published trigger reads near chance on the paper’s labels, while readiness probes decode the labels more strongly; the comparison is deliberately asymmetric and bounded. The section also details validation, intervention design, coverage, and deployment cost.

I.2 VALIDATION: THE SAME HEAD ON ITS OWN BENCHMARK

  • On Charades-STA, the same head reaches R@IoU=0.5 of 53.2 against 42.4 reported in its original paper, validating that it was driven at or above its published level.Scored by the paper’s metric on that task, it reads AUROC 0.6998, versus 0.4837–0.5133 on the paper’s tasks.Source: §I.2 · p. 49

I.3 ZERO-SHOT READINGS ON OUR LABELS

  • On natural ActivityNet-Captions footage, MMDuet’s relevance head reads 0.4837 against readiness at 0.753; on shot identification it reads 0.5133 against readiness at 0.9260.The ActivityNet frame grid matches the window grid with 0.00 s staleness.Source: §I.3 · p. 49
  • On latency-matched answer selection, the probe’s edge over confidence is +0.26759 on shot identification, while MMDuet’s relevance head scores −0.21859 against the same reference.The relevance head is significantly worse than confidence on that comparison.Source: §I.3 · p. 49

I.4 THE TRAINED DIRECTION, ON ITS OWN BASE MODEL

  • On MMDuet’s own adapted stream, its relevance direction reads AUROC 0.6072, while a probe on identical rows and activations reads 0.8433; their cosine is 0.0162.The trained direction falls inside a random-direction cosine null, while readiness is linearly present in that residual stream.Source: §I.4 · p. 50
  • MMDuet’s informative head reads 0.6662 at the same cell, and its two heads have cosine 0.0286; the authors caution that readiness need not be a single thing.The informative-head result is reported as an observation, not as evidence about question-conditioning.Source: §I.4 · p. 50

I.6 DEPLOYMENT COST

  • On the primary geometry, the readiness readout costs 1.057 µs versus 251.24 ms for a monitored forward pass; a separate activation model is estimated at roughly 237,000× the readout cost.The separate-model figure is a proxy, and timing values are wall-clock inference seconds rather than video seconds.Source: §I.6 · pp. 50–51

I.7 WHAT THE COMPARISON DOES AND DOES NOT SUPPORT

  • The comparison supports foreign-trigger transfer failure, not superiority on the trigger’s home task; only MMDuet is validated, and both scored rivals share a Qwen2 backbone.Dispider has no head-level benchmark number to reproduce, and the comparison cannot fully distinguish a weaker state from a different triggered notion.Source: §I.7 · p. 51

J.1 CHOOSING THE SITE

  • The intervention site is selected at mid-depth to avoid layers where the answer letter is already decodable, and the probe must retain AUROC ≥0.80 under a deferral-prompt variant.Configurations below that prompt-stability threshold contribute no results.Source: §J.1 · p. 51

J.3 RESULTS

Adding the readiness direction changes answer commitment at the tested intervention site, but the evidence establishes sufficiency rather than necessity and is limited in scope. The supplied passages also describe evaluation boundaries, reproducibility limits, annotation proxies, and calibration constraints.

J.3 RESULTS

  • On the matched-pair test split, adding the readiness direction changes answer commitment; the second model’s dose curve is not fully monotone.The first model’s response is monotone across doses, exceeds all 35 alternative directions, and stays within the answer-content equivalence margin.Source: §J.3 · p. 52, Table J.1 · p. 52

J.4 HEAD-LEVEL MODULATION

  • Two attention heads named before evaluation rank first and second of 32 for modulating the addition effect, with exact p-value 1/496 = 0.0020.A companion index-free statistic measures the share of total effect mass carried by the named units against a within-item permutation null.Source: §J.4 · p. 52

J.5 SCOPE

  • The intervention establishes sufficiency at one tested site, but not necessity; the evidence covers one matched-pair construction and two video-language models.Subtraction was no more specific than a magnitude-matched wrong-target edit, and the strictest knockout did not remove the natural commitment gap.Source: §J.5 · p. 53

K.1 WHAT IS AND IS NOT FITTED

  • The model remains frozen throughout; only the L2-regularized logistic-regression probe is fitted, and evaluation is group-held-out with nested cross-validation for layer and regularization selection.Confidence intervals use cluster bootstraps over whole groups with 1,000 resamples.Source: §K.1 · p. 53

K.2 DETERMINISM, AND TWO EXCEPTIONS WORTH STATING

  • Per-item activation values are not portable across GPU architectures: maximum per-item difference is 0.113 despite arm-level agreement of 0.0059.The constructed benchmarks use deterministic loaders, while one Oops! option set requires its committed artifact because regenerating it would require a full generator-model pass.Source: §K.2 · pp. 53–54

L.1 COVERAGE

  • The representation evaluation spans seven models, but graded accumulation uses one model, interventions cover two models at one primary site, and cross-attribute transfer uses one model.All evaluated models are open-weight; these narrower results should not be read as having the representation claim’s breadth.Source: §L.1 · p. 54

L.2 EVIDENCE ANNOTATIONS ARE PROXIES

  • Grounding intervals are proxies for earliest answerability: independent raters recover the benchmark evidence label at AUROC 0.681–0.778 while disagreeing with it on 24–30% of screens.Two natural-evidence evaluations share NExT-QA footage, and the controlled composited-cue experiments use model-screened absence and one shared 17-object library.Source: §L.2 · p. 54, §L.3 · p. 55, §L.4 · p. 55

L.5 CALIBRATION

  • Under distribution shift, the carried-over probe has ECE 0.8079 and accuracy 0.1377 at its carried-over threshold, versus 0.8681 at a target-chosen threshold.Threshold repair requires labels; label-free correction helps only under base-rate shift, and knowing that regime may require knowing the target base rate.Source: §L.5 · p. 55

L.6 METHODOLOGICAL CAVEATS

The paper bounds causal claims by answer leakage at late layers and notes that some crossed controls are impossible for growing prefixes. Its timing policies are evaluated mainly through offline replay, while related work distinguishes evidence readiness from answer uncertainty and other timing policies.

L.6 METHODOLOGICAL CAVEATS

  • Late-layer readiness directions can nearly coincide with the direction emitting a fixed answer option, so causal claims use a mid-depth site before the answer letter becomes decodable.Randomizing option order removes the confound and strengthens the mid-depth effect.Source: §L.6 · p. 55
  • A crossed 2 × 2 design cannot be built on a growing prefix because evidence spans start before they end; fixed-span windows permit both time orderings.Therefore, prefix-based crossed evaluations report paired contrasts only, while the reported difference-in-differences is a window result.Source: §L.6 · pp. 55–56
  • The same-pixels control is unavailable on both graded benchmarks: one has no disjoint-evidence question pairs, while the other yields only 22 separable pairs over 20 videos.The fallback for the latter asks a different video’s question on identical frames.Source: §L.6 · p. 56

L.7 OFFLINE REPLAY

  • Accuracy results are offline ladder replays; the live-decoding check does not establish replay equivalence, and selected checkpoints should not be inferred from these experiments.Memory growth over long streams is also unmeasured.Source: §L.7 · p. 56
  • The reported low overhead applies to fixed answer sets and fixed-span windows; open-ended generation and dynamic key-value eviction on longer streams remain untested.For the fixed-answer setting, one forward pass supplies both the answer distribution and the probe input without retained key-value state.Source: §L.7 · p. 56

M EXTENDED RELATED WORK

  • The paper consumes temporal evidence annotations rather than competing with retrieval, but an annotated span need not mark the earliest sufficient evidence.A trained span regressor is not included as a baseline.Source: M EXTENDED RELATED WORK · p. 57
  • Unlike calibration and semantic entropy, readiness is an input-side label defined without the model’s answer, and the streaming policy waits for additional visual evidence rather than allocating computation before prediction.The policy chooses when to answer as evidence arrives or which cached answer to return at a fixed deadline.Source: M EXTENDED RELATED WORK · pp. 57–58
  • The activation intervention establishes sufficiency separately from necessity, using per-example magnitude matching; the reported necessity tests do not support a necessary readiness locus.The authors identify the question-swapped, byte-identical-pixel construction and label independent of model behavior as part of the contribution’s design.Source: M EXTENDED RELATED WORK · p. 57

Related papers

Loading 2610.08560v1…