PaperlayerView exact revision on arXiv

Source-linked AI summary

A Deterministic Evidence Layer for Vision-Language Autism Screening from Naturalistic Home Video

Wenqi Li, Mindi Ruan, Chuanbo Hu, Shuo Wang, Xin Li

arXiv:2610.09217v1cs.CVcs.AI

TL;DR

Specialist observation is central to ASD diagnosis, but access to that expertise is a bottleneck for early identification. This paper keeps the VLM frozen and moves screening decisions to a deterministic evidence layer; on home free-play clips, it improves the share labeled correctly in every run over a zero-shot baseline.

  • Problem

    VLMs can describe behavior in video, but repeated screening runs can produce different labels, while specialist expertise for ASD evaluation is scarce.

    Source: §1 · p. 1

  • Method

    A frozen VLM creates a grounded timestamped event table, a text-only stage age-calibrates its rows, and a deterministic weight-of-evidence scorer assigns risk.

    Source: §1 · p. 2, §3.3 · p. 5, §3.4 · p. 5

  • Results

    74.4% of clips were labeled correctly in every run by the full pipeline, versus 60.5% for the zero-shot baseline.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The decision decomposes into named per-feature contributions and can be re-scored from the saved table without another model call.

    Source: §1 · p. 2

    The study covers 43 clips from one site, including only 12 ASD cases, and its cut-offs were set on this cohort and cannot be validated on it.

    Source: §5 · p. 8

Abstract

from arXiv · show

Autism spectrum disorder (ASD) is diagnosed through specialist observation of a child's social behavior, and access to that expertise is the bottleneck for early identification. Vision-language models (VLMs) describe a child's behavior from video well; the verdict drawn from the description is unstable: at temperature~0, across eight pipeline configurations on one backbone, 16--37\% of clips change their predicted label between repeated runs, and the cause lies in the serving stack. We keep the VLM frozen and move the decision out of the model. Under our grounded perception constraints the VLM writes an event table of timestamped, glossary-labeled events that names the eliciting press and logs counter-evidence; a text-only stage age-calibrates the confidence of each row; a deterministic weight-of-evidence scorer sums it into an evidence total and stratifies it into a risk category, so every decision decomposes into named per-feature contributions and can be re-scored from the saved table. On 43 caregiver-recorded, protocol-free home free-play clips of preschool children, the pipeline reaches AUC $0.851 \pm 0.012$, 86.0\% accuracy, and F$_1$ 71.8 over three runs. It labels 74.4\% of clips correctly in every run (60.5\% for the zero-shot baseline) and flags no typically developing clip in every run (9 of 31 at zero-shot). An ablation on the same backbone attributes the gain to the grounded perception constraints read through the deterministic scorer.

1 Introduction

VLM screening decisions can vary across repeated runs, despite useful video descriptions. The paper separates observation from decision and reports improved stable correctness with a deterministic evidence layer.

1 Introduction

  • The proposed home-video screen is positioned after a positive caregiver-report screen as triage while families wait for evaluation.Specialist expertise is scarce, and the input is caregiver-supplied, protocol-free home video.Source: §1 · p. 1
  • At temperature 0, 16–37% of clips changed labels across repeated runs, motivating stable correctness as the target.Stable correctness is the share of clips labeled correctly in every run.Source: §1 · pp. 1–2
  • The pipeline keeps the VLM frozen for timestamped event observations, age-calibrates rows, then uses a deterministic scorer to assign risk categories.The saved table supports per-feature explanations and rescoring without another model call.Source: §1 · p. 2
  • Grounded perception constraints require interval-bound rationales, named eliciting presses for social-response deficits, and explicit typical-behavior counter-evidence.In the cumulative ablation, specificity rose from 91.4% to 94.6%, F1 from 50.7 to 71.8, and accuracy from 77.5% to 86.0%.Source: §1 · p. 2
  • 74.4% of clips were labeled correctly in every run by the full pipeline, versus 60.5% for the zero-shot baseline.No typically developing clip was flagged in every run, compared with 9 of 31 at zero-shot.Source: §1 · p. 2

2 Related Work

Prior work includes engineered or learned video features, supervised systems, and VLM-based approaches to autism screening. This paper situates its contribution at the intersection of decomposed medical reasoning and repeatability of clinical model decisions.

Screening ASD from video

  • Home-video screening research spans engineered behavioral features, learned models, supervised systems, and two VLM neighbors using zero-shot scoring or fine-tuning.The paper distinguishes its approach by using no elicitation protocol, no training anywhere in the pipeline, and a private evaluation corpus.Source: §2 · p. 3

Decomposed and verifiable medical reasoning

  • Medical reasoning systems have separated perception from judgment and reserved final decisions for deterministic or statistical layers; this paper applies that separation to raw, unprotocolized home video.Its interface is a probe-gated timestamped event table with explicit counter-evidence under a never-erase rule.Source: §2 · p. 3

Reliability of clinical (V)LM decisions

  • Prior clinical-model reliability studies report repeated-output instability and remedies including prompting, sampling, or ensembling, while this paper measures clip-level label flips and uses a fixed scorer on saved event tables.The cited serving-stack mechanism is not fixable from the user side of a closed API.Source: §2 · p. 3

3 Method

The method separates video-based observation from age-aware interpretation and a deterministic decision layer. It produces a calibrated event table, evidence total, and risk category that can be re-scored from the saved table.

3.1 Clinical Cohort and Home-Video Corpus

  • The evaluation set contains 43 clips (12 ASD, 31 TD) of caregiver-filmed, unprompted 3–6-minute home free play, with clip-level labels confirmed by independent ASD specialists.Expert-guided annotations ground an 11-item glossary; recordings are not publicly distributed.Source: §3.1 · p. 4

3.2 Problem Formulation

  • The screening function maps video and age to a calibrated event table, evidence total, and risk category through two fixed model calls followed by a deterministic decision layer.The perception model sees video; the text-only judgment model never sees it.Source: §3.2 · p. 4

3.3 Stage 1: Perception into a Grounded Event Table

  • Stage 1 logs behaviors with timestamps, observable descriptions, glossary labels, context, and confidence; rationales are interval-bound and response-deficit coding requires a named adult press.Typical behaviors are recorded as counter-evidence, while Stage 1 logs raw social material rather than coding pervasive feature #4.Source: §3.3 · pp. 4–5

3.4 Stage 2: Text-Only Age Calibration Under the Never-Erase Rule

  • Stage 2 checks counter-evidence before synthesizing feature #4, age-calibrates remaining rows, and re-emits the table without deleting rows or raising confidence.It may lower confidence, add an age note, or add feature #4; it cannot relabel a coded feature as typical.Source: §3.4 · p. 5

3.5 Deterministic Weight-of-Evidence Scorer

  • Social-communication evidence is capped at 1.5 nats while restricted/repetitive evidence sums in full; cut-offs at 1.5 and 1.8 define low, intermediate, and high risk.Each feature contributes its log positive-likelihood-ratio weight once, at its highest observed confidence.Source: §3.5 · pp. 5–6

4 Experiments and Results

The experiments evaluate three pipeline configurations on a 43-clip test set, then examine classification performance, cumulative ablations, and correctness across repeated runs.

4.1 Experimental Setup

  • Evaluation used 43 test clips (12 ASD, 31 TD), three seeds per configuration, and temperature 0; the full pipeline uses constraints and a deterministic scorer.The zero-shot baseline uses a single VLM verdict, while the two-stage split uses a text-only Stage 2 verdict without grounded constraints or the scorer.Source: §4.1 · p. 6

4.2 Classification Performance

  • Three of 12 ASD clips stayed below the screen-positive cut-off in every run, while no TD clip was flagged in more than one run.Eight clips crossed the screen-positive cut-off in some runs only; three consistently missed ASD clips had near-empty event tables.Source: §4.2 · p. 6
  • AUC 0.851 ± 0.012 and accuracy 86.0% ± 0.0% were achieved by the full pipeline on 43 clips; Figure 2A shows its ROC curve.The reported AUC uses the scorer’s evidence total; the figure shows the mean and individual runs.Source: §4.2 · p. 6, Figure 2 · p. 7

4.3 Ablation Study

  • Accuracy rises from 69.0% at zero-shot to 77.5%, 81.4%, and 86.0% as modules accumulate; the final constraints-and-scorer rows reach F1 71.8.Across the last three rows, recall rises from 41.7% to 47.2% to 63.9%, while specificity stays at 91.4–94.6%.Source: §4.3 · p. 7, Table 1 · p. 7

4.4 Stable Correctness Across Repeated Runs

  • pass3 rises from 60.5% at zero-shot to 74.4% for the full pipeline, while 16.3%–37.2% of clips flip labels across eight configurations.pass3 counts clips labeled correctly in all three runs; Figure 3 contrasts it with pass@1 for each configuration.Source: §4.4 · pp. 7–8, Figure 3 · p. 8
  • Nine TD clips are flagged in every zero-shot run and none from the two-stage split onward; the full pipeline misses three ASD clips in every run.The scorer recovers two clips missed by the two-stage split because their event tables carry at least 1.5 nats in each run.Source: §4.4 · p. 8

5 Discussion and Conclusion

The reported operating point suits confirmatory triage in an already-referred population, but prevalence and recall limit its reach. The cohort and missed clips also bound what the results establish.

5 Discussion and Conclusion

  • At 27.9% ASD prevalence, 94.6% specificity gives precision 0.80 (8 of 10 flagged clips in one run), while 63.9% recall means a negative result cannot exclude ASD.The authors note that precision would fall sharply at general-population prevalence.Source: §5 · p. 8
  • Grounded perception constraints read through the scorer carry the ablation gain, while three ASD clips missed in every run set the current recall ceiling at 75.0%.The authors identify better grounded observation, including prosody and caregiver speech, as a next step for under-detected reciprocity features.Source: §5 · p. 8
  • Leave-one-out re-selection gives pooled recall of 58–61% at 91–95% specificity, versus 47.2% at 94.6% on the same tables, and ranking beats age alone.The high-risk cut-off is the rule-in end of a dial; the same saved tables can be re-scored at a lower cut-off for a rule-out screen.Source: §5 · p. 8
  • The 43-clip, one-site study has 12 ASD cases and a wide DeLong interval ([0.72, 1.00]); evidence missed in Stage 1 cannot be recovered downstream.The authors call for external validation at a second site and prospective studies in which clinicians act on the tables.Source: §5 · p. 8

A Cohort Composition

Table A1 describes the home-video corpus, with ages recorded at the time of recording and four demonstration clips excluded from reported metrics.

A Cohort Composition

  • The four in-context demonstration clips are excluded from every reported metric, and the listed ages are children’s ages at recording.The table is identified as the corpus composition.Source: Table A1 · p. 14

B Scorer: Derivation and Design Rationale

The scorer uses a Bayesian weight-of-evidence rationale, with design choices for unobserved behaviors, uncertain events, correlated features, and the provenance of its feature estimates.

B Scorer: Derivation and Design Rationale

  • The derivation applies Bayes’ rule under conditional independence of features given ASD or TD, with E denoting observed glossary features.The scorer keeps the positive log-likelihood-ratio sum over observed features and drops the absence term because an unprompted clip may not provide an opportunity for a behavior.Source: B Scorer: Derivation and Design… · p. 14
  • Possible events contribute one quarter of their weight, and each feature counts only once at its highest observed confidence.This fixed discount applies to occluded, brief, or age-downgraded observations; repeated events cannot accumulate.Source: B Scorer: Derivation and Design… · p. 14
  • The social-communication total is capped at 1.5 nats to limit double-counting correlated features, while near-independent restricted and repetitive behaviors are summed in full.The cap is described as about one strong deficit, with LR+ approximately 4.5.Source: B Scorer: Derivation and Design… · p. 14
  • Eleven author-estimated sensitivity/specificity pairs and cohort-derived cap and cut-offs determine feature weights and risk thresholds; feature 8 is the least secure pair.The pairs were fixed before reported runs, and feature 8 did not separate groups in a toddler cohort.Source: B Scorer: Derivation and Design… · p. 14

B.1 Leave-one-out re-selection of the cap and the cut-offs

Leave-one-out re-selection tests the fixed decision rule by choosing the cap and cut-off on 42 clips and applying them to the held-out clip. Re-selection outcomes vary across runs, while cap removal has limited AUC impact.

B.1 Leave-one-out re-selection of the cap and the cut-offs

  • The fixed cap and cut-offs were set on an earlier run over this cohort before the three reported runs.The screen-positive cut-off is 1.5 nats and the high-risk cut-off is 1.8 nats.Source: §B.1 · p. 14
  • Table A2 lists assumed sensitivity and specificity for each feature, its weight in nats, and the evidence type used as its basis.The weight is defined as wf = ln(sensf/(1 −specf)).Source: Table A2 · p. 15
  • Joint leave-one-clip-out selection yields pooled recall of 58.3–61.1% and specificity of 91.4–94.6%, versus 63.9% and 94.6% at the deployed cut-off.Across folds and criteria, the selected cap ranges from 0.5 to 1.75 nats.Source: §B.1 · p. 15, Table A3 · p. 15
  • The Youden cut-off is unstable across runs: its choice between two levels turns on four clip-runs.The four include two TD clips at the cap, one TD clip at 1.69 nats, and one ASD clip reaching the cap in one run.Source: §B.1 · p. 15
  • Because the screen-positive cut-off equals the cap, removing the cap cannot change screen-positive decisions; it moves two TD clip-runs from intermediate to high risk and changes AUC by at most 0.008.The reported AUC changes are 0.863 to 0.855, 0.839 to 0.839, and 0.851 to 0.853.Source: §B.1 · p. 15

C Cross-Seed Flip Rates by Configuration

Table A4 reports cross-seed flip rates for Gemini 3.1 Pro across configurations, measuring how often clip labels differ between repeated runs at temperature 0.

C Cross-Seed Flip Rates by Configuration

D Run-to-run Stability of the Single-Call Baseline

The single-call baseline was repeated five times using an identical request at temperature 0. Table A5 reports run-to-run stability for that baseline over 43 clips.

D Run-to-run Stability of the Single-Call Baseline

E Assets: Glossary, Prompt Structure, Norms, and Demonstrations

The appendix documents the prompts, behavioral glossary, age-referenced norms, and demonstrations used by the two-stage pipeline. Together, these assets specify the glossary and age-calibration materials and show how the stages exchange and revise event tables.

E.2 Behavioral glossary

  • The 11-item glossary condenses a 22-behavior, nine-task home-video protocol into behaviors codeable from unprompted free play.Its features include ADOS-2 / DSM-5 mapping where applicable, behavioral descriptions, typical contexts, and examples, and constrain both expert annotation and model prompting.Source: §E.2 · p. 16, Table A6 · p. 18

E.1 Prompt structure

  • Figure A1 shows each stage’s system prompt and its demonstrations, with Stage 1’s event table passed into Stage 2’s final turn.The prompts include glossary and norms knowledge blocks; demonstrations pair prior user and assistant turns.Source: §E.1 · p. 16, Figure A1 · p. 17

E.3 Age-referenced norms

  • Stage 2 downgrades age-typical behaviors to possible with an age note, reserving certain for behavior exceeding age expectations in intensity, duration, or pervasiveness.Pervasiveness overrides age-based exemptions when behavior recurs or dominates the clip.Source: §E.3 · p. 19
  • Figure A2 maps each feature to ages when its behavior is typical, transition ranges, conditions for retaining it, and developmental anchors.Pervasiveness overrides the age window at any age.Source: Figure A2 · p. 19

E.4 Stage 2 demonstrations

  • The two Stage 2 demonstrations use user turns with the child’s age and input table, followed by assistant output turns.The input tables are identical except for the changes shown in the outputs.Source: §E.4 · p. 19
  • Figure A3 presents two hand-written Stage 2 examples at 30 and 42 months, with age notes and feature #4 changes marked.The examples illustrate an overture blocking the pervasive feature in one case and its addition after a counter-evidence check in the other.Source: Figure A3 · p. 20
Loading 2610.09217v1…