Source-linked AI summary
PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment
Fan Yuxuan, Huang Miaojun, Zhang Haimei, Wu Jingshen, Liu Hao
TL;DR
Interview assessment needs evidence-grounded criterion judgments, but growing applicant volumes make human-only evaluation costly and inconsistent, while existing AI systems can produce opaque scores. PhoenixNest-Video uses a semantic video graph, rubric-conditioned cross-modal retrieval and verification, and rubric-trained scoring to anchor judgments to candidate materials. It achieves 91.50% grade-level accuracy on VInterview-2025 and outperforms substantially larger baselines while exposing evidence for review.
Problem
Rising interview volumes make rigorous human-only assessment costly and inconsistent, while existing AI approaches lack traceable rationale and struggle with criterion-level multimodal judgment.
Method
PhoenixNest-Video builds a semantic video graph, retrieves and verifies rubric-relevant evidence across modalities, and trains scoring with dual rewards for rubric alignment and score-level differentiation.
Results
91.50% grade-level accuracy on VInterview-2025 was achieved with the lowest total-score MAE and Wasserstein distance among baselines, while outperforming substantially larger proprietary models.
Takeaways & Limitations
Rubric grounding and evidence traceability support compact multimodal agents for scalable, transparent, and auditable structured interview assessment.
Takeaways & Limitations
English-language interviews and imperfect ASR can penalize content-related criteria through language proficiency and transcript-quality confounds.
Abstract
from arXiv · showhide
Interview assessment requires per-criterion judgments grounded in behavioral evidence, yet surging applicant volumes have made human-only evaluation costly and inconsistent, while existing AI approaches yield opaque scores without traceable rationale. We introduce PhoenixNest-Video, an evidence-grounded multimodal agent framework for automated video interview assessment. It builds a semantic video graph as structured working memory, performs rubric-conditioned retrieval with cross-modal verification across visual, audio, and textual streams, and produces per-criterion scores anchored to the candidate's materials. A Scorer trained via Rubrics-based Reinforcement Learning with dual rewards for rubric alignment and score-level differentiation internalizes the discriminative structure of multi-level rubrics. PhoenixNest-Video attains 91.50\% grade-level accuracy on VInterview-2025, outperforming substantially larger proprietary models. A compact, rubric-grounded agent therefore scores candidates in closer agreement with an expert panel than direct prompting of much larger models, and exposes the evidence behind each score for human review.
1 Introduction
Interview assessment must produce criterion-level judgments justified by specific evidence, but rising volumes make human-only evaluation costly and inconsistent. PhoenixNest-Video addresses this gap with rubric-grounded, evidence-traceable multimodal scoring.
- Application volumes rose over 37% between 2021 and 2025 to more than 7.6 million submissions, increasing pressure on evidence-anchored assessment.
- Human assessment is accurate but difficult to scale because expert staffing, labor costs, fatigue, anchoring bias, and inconsistent criterion weighting constrain large-scale evaluation.
- ML-based systems scale but produce opaque scores, while general-purpose MLLMs struggle to organize criterion-relevant multimodal evidence and differentiate candidates along rubric-defined dimensions.
- PhoenixNest-Video combines a semantic video graph, rubric-conditioned retrieval, cross-modal verification, and evidence-anchored per-criterion scoring.
- 91.50% grade-level accuracy on VInterview-2025 was achieved alongside the lowest total-score MAE and Wasserstein distance among baselines, despite an 8B backbone.
- The paper introduces Rubrics-based Reinforcement Learning with dual rewards for rubric alignment and score-level differentiation, and reports score distributions closely aligned with human experts.
2 Related Work
Automated video interview assessment has progressed from handcrafted unimodal features to multimodal benchmarks and naturalistic datasets. PhoenixNest-Video targets the remaining need for evidence-grounded, traceable reasoning in structured evaluations.
- Earlier systems used handcrafted prosodic and facial features, then advanced through bimodal personality prediction and hierarchical neural models for asynchronous screening.
- Recent work broadens assessment to audio-visual personality, pose, multimodal performance, naturalistic interviews, and fairness-related settings.
- Long-horizon temporal reasoning remains a bottleneck, distinguishing PhoenixNest-Video’s focus on evidence-grounded assessment and traceable reasoning.
3 Task Formulation
The task models interview assessment as independent criterion scoring with aggregation across panelists. PhoenixNest-Video formalizes this process as mapping video, optional materials, criteria, and rubrics to scores, evidence references, and feedback.
- In the typical protocol, candidates present to a panel, panelists independently rate predefined criteria, and final outcomes aggregate their score vectors.
- The evaluation function maps a candidate’s video, optional supplementary materials, criteria, and score-level rubric descriptors to per-criterion scores, evidence references, and textual feedback.
- The formulation treats the number of criteria, score scale, and rubric content as application-supplied parameters and evaluates configurations across two benchmarks.
4 Methodology
PhoenixNest-Video converts interviews and rubrics into structured multimodal evidence, then retrieves, verifies, and scores criterion-level performance with a trained scorer. Its pipeline combines semantic graph memory, rubric-grounded retrieval, cross-modal evidence verification, and reinforcement learning for rubric alignment and score differentiation.
- 4.1 Multimodal Preprocessing: PhoenixNest-Video preprocesses interviews into structured visual, audio, transcript, and slide streams, while expanding rubric descriptors into behavioral indicators.The visual stream samples 32 frames per video and extracts slide images; behavioral indicators are generated for each criterion and score level.
- 4.2 Pipeline Components: The Grapher partitions videos into clips and builds a semantic graph linking clips that share prototype entities or equivalent content.Clips are sampled at 1.0 FPS with K = 64 frames, and mentions are extracted with aligned transcripts; shared prototype entities connect temporally separated moments.
- 4.2 Pipeline Components: Rubric-conditioned retrieval uses behavioral indicators and semantic keywords to select criterion-relevant clips for subsequent evidence verification.The behavioral indicators both ground training rewards and direct retrieval queries during inference.
- 4.2 Pipeline Components: The Scorer uses the reasoning chain, rubric, and behavioral indicators to produce predicted scores, rationales, feedback, and references to candidate materials.Its output is intended to serve directly as the final assessment report.
- 4.3 Scorer Training: The Scorer is trained with supervised fine-tuning followed by rubrics-based reinforcement learning using alignment and differentiation rewards.The alignment reward evaluates rubric-faithful reasoning and evidence support, while the differentiation reward compares predicted and expert total-score grading levels.
- 4.3 Scorer Training: PhoenixNest-Video reports total-score alignment and per-criterion agreement against proprietary, open-source, and video-specific baselines on VInterview-2025.Table 1 distinguishes institutional grading-level total-score metrics from agreement on individual 0–2 rubric scores and excludes missing or non-parseable outputs.
5 Experiments
Experiments evaluate PhoenixNest-Video against diverse MLLM baselines, test component contributions, assess transfer to a different annotation schema, and analyze fairness and frame-sampling sensitivity. Across these evaluations, the framework achieves strong assessment performance, benefits from its rewards and retrieval-verification components, and remains effective on RecruitView.
- 5 Experiments: The experiments compare PhoenixNest-Video with proprietary, open-source, and video-specialized MLLMs under a common prompt-based protocol.They also test whether the framework improves large MLLM backbones beyond direct prompting.
- 5.2 Main Results (RQ1): Each PhoenixNest-Video-wrapped backbone improves over its direct-prompt counterpart on every reported metric.The framework therefore functions as a backbone amplifier rather than relying only on backbone scale.
- 5.3 Ablation Studies (RQ2): Removing either reward signal, the Verifier, or reinforcement learning degrades accuracy and increases MAE, with SFT-only training producing the largest regression.Ralign supports rubric-faithful rationales, Rdiff shapes institutional-level score differentiation, and verification contributes independently.
- 5.4 Results on RecruitView (RQ3): On RecruitView, PhoenixNest-Video achieves the best score across all four macro-averaged rank-correlation and concordance metrics over 12 continuous targets.The result holds under a different dataset, target structure, and native train/test protocol.
- 5.5 Deep Analysis (RQ4): Frame-sampling accuracy rises sharply from 10 to 16 frames, plateaus through 32, and degrades at 64, making 32 frames the default.The analysis attributes the decline at higher density to visual-token inflation and diluted attention across redundant frames.
6 Conclusion
PhoenixNest-Video is an evidence-grounded multimodal agent for automated video interview assessment. Its rubric-grounded scoring and evidence traceability outperform substantially larger baselines on two benchmarks.
- PhoenixNest-Video combines Rubrics-based Reinforcement Learning with dual rewards and a four-stage pipeline that anchors scores to cross-modally verified candidate evidence.The dual rewards target rubric alignment and score-level differentiation.
- On VInterview-2025 and RecruitView, PhoenixNest-Video outperforms substantially larger proprietary and open-source baselines.
- The framework’s results indicate that rubric grounding and evidence traceability matter more than raw model scale for structured assessment.
Limitations
The reported fairness audit and language-related findings apply within limited data and institutional settings. Unmeasured attributes, communication conditions, transcript errors, fixed ordering, offline assessment, and restricted data access constrain broader deployment.
- Scope of the fairness audit: The fairness audit covers gender and academic discipline in one admissions cohort at one institution, with group sizes detecting only large disparities.Other candidate attributes were not recorded, and retrieved signals may reflect cultural norms or unequal coaching and recording access.
- Language proficiency and transcript quality: English-language interviews and imperfect recordings can degrade ASR transcripts and depress content-related criteria for less-fluent candidates.The Scorer reproduces this confound from faculty ratings; separating language proficiency from target competencies requires more varied data.
- Fixed design choices and deployment scope: Chronological reasoning-chain inputs are fixed and may shift scores under alternative ordering, while streaming assessment remains unaddressed.The system assumes the full interview is available before assessment begins.
- Fixed design choices and deployment scope: VInterview-2025 remains closed because identifiable recordings cannot be anonymized sufficiently for release.The corpus is retained on institutional infrastructure, with neither recordings nor the derived benchmark released.