Source-linked AI summary
UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning
Lei Xin, Zeheng Wang, Jiayin Zhu, Shihong Huang, Fanhu Zeng, Changjiang Jiang, Dengbo He, Yutao Yue, Zhenglun Kong
TL;DR
ASD recognition methods rely heavily on visual analysis despite limited clinical-semantic descriptions and scarce paired visual-text data. UniAR generates hierarchical diagnostic descriptions and aligns them with visual evidence; it consistently outperforms strong baselines across MRI and facial benchmarks.
Problem
ASD recognition often relies on visual analysis, while clinically meaningful semantic descriptors and paired visual-text training data are limited.
Source: §1 · pp. 1–2Method
UniAR generates word-, phrase-, and sentence-level diagnostic descriptions and aligns these hierarchical semantics with visual evidence.
Source: §1 · p. 2, §3.3 · p. 4Results
Across MRI and facial benchmarks, UniAR consistently outperforms strong baselines, with average accuracies of 75.9% on MRI benchmarks and 91.6% on facial benchmarks.
Source: §1 · p. 2Takeaways & Limitations
UniAR offers a practical way to bridge low-level biomarkers and high-level diagnostic reasoning, with reported benefits under heterogeneous MRI settings, subtle facial behavior variations, and incomplete diagnostic text conditions.
Source: §5 · p. 8Grad-CAM does not provide direct causal evidence for each semantic level.
Source: §4.4 · p. 7
Abstract
from arXiv · showhide
Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term developmental outcomes. However, existing ASD recognition methods are often constrained by the scarcity of diagnostic text data, forcing them to rely mainly on visual analysis and limiting their ability to model clinically meaningful semantic reasoning. To address this challenge, we propose UniAR, a unified framework enhanced by multi-granularity prompt learning for robust ASD recognition under heterogeneous data variations. Specifically, UniAR leverages a large multimodal model to generate hierarchical diagnostic descriptions at the word, phrase, and sentence levels, compensating for the lack of paired clinical reports. To align the generated semantics with visual evidence, we further design a Mixture-of-Experts-based Multi-Scale Alignment Module, which dynamically matches vector-quantized visual prototypes with semantic representations at corresponding granularities. Extensive experiments on four benchmarks covering brain MRI and facial expression scenarios show that UniAR consistently outperforms existing state-of-the-art methods, achieving average accuracies of 75.9\% on MRI benchmarks and 91.6\% on facial benchmarks, while improving average Accuracy on MRI benchmarks by 1.5 percentage points and average Accuracy on facial benchmarks by 1.2 percentage points over baselines. These results demonstrate that UniAR offers a robust and interpretable framework for ASD screening under semantic scarcity.
1 Introduction
ASD screening faces heterogeneous symptoms, limited clinical-semantic modeling, and scarce paired visual-text data. UniAR synthesizes hierarchical descriptions, aligns them with visual features, and reports strong results across MRI, facial, and social-media evaluations.
1 Introduction
- Clinical scales and parent reports can struggle with ASD presentation heterogeneity, while visual-only recognition overlooks semantic descriptors such as emotional responses and social interactions.Early identification and intervention during early childhood are described as instrumental to improving long-term developmental outcomes.Source: §1 · pp. 1–2
- Paired visual-text data are scarce and costly to obtain, and privacy restrictions on diagnostic text limit data sharing for multimodal learning.The introduction links these constraints to difficulties realizing multimodal methods' potential for accurate and practical diagnostic systems.Source: §1 · p. 2
- UniAR generates diagnostic descriptions at word, phrase, and sentence levels and aligns them with pixel-level, regional, and global visual features.The framework links diagnostic semantics with behavioral biomarkers for multi-scale visual-semantic fusion.Source: §1 · p. 2
- 75.9% average accuracy on MRI benchmarks and 91.6% on facial benchmarks; UniAR consistently outperforms strong baselines and improves over EAC across evaluated social platforms.It also achieves the best performance on ABIDE I and HRM.Source: §1 · p. 2
- UniAR combines hierarchical prompting, semantic-guided modeling, and codebook-based prototype refinement to support robust, interpretable ASD recognition under limited data.The introduction also evaluates a cross-platform social-media setting alongside public MRI and facial-expression benchmarks.Source: §1 · p. 2
2 Related Work
Prior ASD recognition work strengthens neuroimaging, behavioral, or multimodal representations, while explicit clinical-semantic modeling remains limited. Related work on missing modalities and medical vision-language alignment often assumes paired reports or curated disease descriptions.
ASD Recognition
- ASD recognition methods span connectome modeling, behavioral-cue analysis, and multimodal fusion, but explicit clinical-semantic modeling remains limited.Neuroimaging work includes resting-state or structural biomarkers, while facial and behavioral methods examine cues such as head pose, expressions, and social responses.Source: §2 · p. 2
Semantic Reconstruction and Missing-Modality Learning
- Augmentation, modality completion, and prompt-based adaptation address limited supervision or missing inputs, while diagnosis-level semantic reconstruction without clinical text remains underexplored.Examples include pseudo-sample synthesis and enriched feature distributions for small-data ASD settings.Source: §2 · p. 3
Medical Visual-Language Alignment
- Medical vision-language methods align reports with regions, instances, entities, or image patches, but generally assume paired reports or curated disease descriptions during training.The surveyed approaches progress from report-level matching toward finer-grained and knowledge-guided alignment.Source: §2 · p. 3
3 The Proposed Method
UniAR formulates ASD recognition as supervised classification while reconstructing missing diagnostic semantics from visual inputs. Its pipeline stabilizes multi-scale visual features, generates hierarchical descriptions, and aligns corresponding scales for prediction.
3.1 Preliminaries
- The supervised classification objective uses visual inputs and generated semantics, combining classification, alignment, hierarchical regularization, fusion, and prototype-learning losses.The formulation uses a large multimodal generator to approximate unavailable clinical semantics.Source: §3.1 · p. 3
3.2 Framework Overview
- The three-stage pipeline builds codebook-based multi-scale visual features, generates hierarchical descriptions, then aligns matched scales for ASD prediction.UniAR first stabilizes visual representations, then reconstructs missing semantics, and finally addresses the scale mismatch through explicit alignment.Source: §3.2 · p. 4
3.3 Multi-Granularity Visual-Semantic Representation Construction Module
- UniAR uses a learnable visual codebook to map features to discrete prototypes and GPT-4o to generate word-, phrase-, and sentence-level diagnostic descriptions.Prototype quantization suppresses instance-specific noise; the description levels capture isolated attributes, local interactions, and holistic diagnostic context.Source: §3.3 · p. 4
3.4 Multi-Scale Alignment Module
- MSAM combines shared and scale-specific experts with cross-attention to modulate matched visual-semantic branches and inject diagnostic semantics into visual features.The experts preserve cross-scale commonality while emphasizing granularity-relevant attributes; visual features serve as queries and semantic embeddings as keys and values.Source: §3.4 · pp. 4–5
4 Experiments
Across MRI and facial benchmarks, UniAR compares favorably with existing methods. Ablations, qualitative visualizations, and cross-platform evaluation examine its components, attention patterns, and performance beyond public benchmarks.
4.1 Experimental Setting
- Evaluation spans ABIDE I/II MRI, HRM/Kaggle facial benchmarks, and ASD-MM, a three-platform, expert-annotated four-class video benchmark.ASD-MM uses Normal, Mild, Moderate, and Severe labels and includes expert-written descriptions covering social competence, language expression, and behavior.Source: §4.1 · pp. 5–6, Figure 5 · p. 7
4.2 Comparison Results
- UniAR ranks first on all four ABIDE metrics and leads facial benchmarks; it reaches 91.6% average Accuracy versus EAC's 88.8% and GPT-4o's 90.4%.On ABIDE II it has the best Accuracy and AUC; on facial benchmarks it leads both dynamic and static settings.Source: Table 1 · p. 6, Table 2 · p. 6, §4.2 · p. 6
4.4 Qualitative Analysis and Visualization
- Grad-CAM visualizations show UniAR's facial responses are more concentrated on clinically relevant areas and more spatially coherent than EAC's.The analysis notes that warmer colors indicate higher relevance and reports less attention leakage to irrelevant background regions.Source: Figure 4 · p. 7, §4.4 · p. 7
4.3 Ablation Study
- Sequential word-, phrase-, then sentence-level alignment improves MRI results, and adding fusion yields the full model's best overall performance.The sentence-level addition improves on word-plus-phrase alignment; similar observations are reported on facial benchmarks.Source: §4.3 · p. 7, Table 3 · p. 8
- 128 codebook entries perform best across four datasets; four MoE experts achieve the best overall results and improve performance across most datasets over two.The paper reports limited or inconsistent gains from a 256-entry codebook and less routing flexibility with smaller expert configurations.Source: §4.3 · p. 7, Table 4 · p. 8, Table 5 · p. 8
4.5 Cross-Platform Quantitative Validation and Interpretable Analysis
- UniAR outperforms EAC across evaluated ASD-MM platforms in Accuracy, F1-score, and Recall, with favorable ROUGE consistency against expert reports.The evaluation addresses cross-platform distribution shifts and whether generated descriptions remain semantically consistent with expert-associated annotations.Source: §4.5 · p. 8
5 Conclusion
UniAR addresses semantic scarcity in ASD recognition by reconstructing hierarchical diagnostic semantics and aligning them with visual evidence. The paper concludes that structured semantic supervision and prototype-based refinement support robustness and interpretability across heterogeneous settings.
5 Conclusion
- Figure 6 compares cross-platform performance, and Table 6 lists evaluation metrics for cross-platform datasets.Source: Figure 6 · p. 8, Table 6 · p. 8
- UniAR combines prototype-based visual representation refinement with hierarchical visual-semantic alignment to improve robustness and interpretability.The framework aims to bridge low-level biomarkers and high-level diagnostic reasoning.Source: §5 · p. 8
- The paper reports structured semantic supervision as beneficial for robust ASD recognition, especially under heterogeneous MRI settings, subtle facial behavior variations, and incomplete diagnostic text conditions.Source: §5 · p. 8