AVI-Personality: A Trait-Activated Multimodal Dataset for Personality and Competency Assessment in Asynchronous Video Interviews

• Existing datasets often use task-irrelevant public videos and simplified or crowdsourced personality labels, weakening trait-measurement and construct validity.Source: §1 · pp. 1–2
• AVI-Personality contains 3,876 interview videos from 646 participants completing a simulated job application with two generic and four HEXACO-targeted questions.Source: §1 · p. 2
• The dataset combines trait-activated interview responses with self-reports, BARS-based observer ratings by trained psychologists, recruiter-rated competencies, interview performance, and cognitive ability.Source: §1 · p. 2
• Annotation validation examined reliability, construct validity, internal nomological association, and demographic fairness, supporting annotation consistency and practical relevance.Source: §1 · p. 2
• Text-based methods provide strong personality-relevant cues, while multimodal methods achieve the best overall performance but only modestly outperform strong text-based baselines.Source: §1 · p. 2

• Personality and competency assessment matters in personnel selection, while questionnaire-based assessment is interpretable but vulnerable to social desirability bias.Source: §2.2 · p. 3
• Automatic assessment has progressed from hand-crafted multimodal features and traditional machine learning toward models requiring rich behavioral signals and reliable labels.Source: §2.2 · p. 3
• Existing datasets commonly use short social-media videos or other weakly structured scenarios and crowd ratings, limiting theoretically grounded personality and competency assessment.Source: §2.3 · pp. 3–4
• Trait Activation Theory motivates structured questions that elicit behavioral expressions linked to theoretically relevant personality traits and workplace behaviors.Source: §2.3 · p. 4, Figure 1 · p. 4
• AVI-Personality combines validated self-report questionnaires with multi-perspective observer ratings to support personality assessment and workplace-related analysis.Source: §2.3 · p. 4

• 646 participants remained after excluding incomplete responses, missing consent, failed attention checks, abnormal HEXACO patterns, low effort, corrupted audio, and rater-identified noncompliance.Source: §3.1 · p. 4
• The final sample included 309 men, 309 women, and 28 non-binary participants, with a mean age of 36.69 years and 15.96 years of work experience.Source: §3.1 · p. 5
• Participants were native English-speaking adults living in the United States, and the sample included multiple reported ethnic groups.Source: §3.1 · p. 5

• The structured interview used generic questions to elicit broad selection information and personality questions to target personality-relevant content.Source: §3.2 · p. 5
• Questions followed a fixed order, with generic questions presented before personality questions; Table 1 records their content, order, types, and corresponding traits.Source: §3.2 · p. 5, Table 1 · p. 5
• An initial pool of 86 literature-based job interview questions was screened to retain 61 open-ended, broadly applicable questions conveying personality information.Source: §3.2.1 · p. 5
• Seventeen professional recruiters evaluated practical usage, achieving ICC(2, 17) = 0.88 inter-rater agreement before expert assessment.Source: §3.2.1 · p. 5
• Four personality experts selected one past-behavior question for each of Honesty-Humility, Extraversion, Agreeableness, and Conscientiousness after resolving disagreements by consensus.Source: §3.2.2 · p. 6

• Participants applied for a fictitious management traineeship and completed an AVI using a study-developed platform.Source: §3.3 · p. 6
• The AVI included two generic questions and four personality questions designed in Section 3.2.Source: §3.3 · p. 6
• Participants were instructed to answer each interview question within 1–2 minutes.Source: §3.3 · p. 6

• The HEXACO-100 measured Honesty-Humility, Emotionality, Extraversion, Agreeableness, Conscientiousness, and Openness to Experience.Items used a 5-point Likert scale, and domain scores were calculated by averaging corresponding items.Source: §3.4.1 · p. 6
• The ICAR assessed general cognitive ability through verbal reasoning, series, matrix reasoning, and three-dimensional rotation tasks.Responses were aggregated into an overall cognitive ability score.Source: §3.4.2 · p. 6

Personality annotation

• At least three of twelve trained personality psychologists rated each participant’s observer-reported personality traits.Raters completed nine hours of training.Source: §3.5.1 · p. 6
• Generic questions elicited ratings for all six HEXACO traits without targeting a specific trait.Source: §3.5.1 · p. 6
• Personality questions elicited ratings for Honesty-Humility, Extraversion, Agreeableness, and Conscientiousness through trait-relevant behaviors.These questions were designed according to Trait Activation Theory.Source: §3.5.1 · p. 7
• Using both question types enabled comparison between non-activated and activated assessment conditions.Source: §3.5.1 · p. 7

Rating procedure

• Personality raters used behaviorally anchored rating scales with scores from 1, Very low, to 5, Very high.Source: §3.5.1 · p. 7

Competency annotation

• At least two professional recruiters rated each participant on four competencies and overall interview performance after reviewing all six questions.The competencies were Integrity, Collegiality, Social versatility, and Development orientation.Source: §3.5.2 · p. 7
• The five competency and interview-performance variables loaded on a single principal component in exploratory PCA.Source: §3.5.2 · p. 7

Reliability

• Mixed-effects models separated participant, rater, and residual variance to estimate observer-rating reliability.Source: §4.1 · pp. 7–8
• MAICC treated systematic rater differences as measurement error, whereas MCICC assessed rank-order similarity after accounting for rater differences.Source: §4.1 · p. 8
• Observer-rated personality traits showed moderate to high reliability, with higher reliability for personality-targeted than generic questions.Extraversion had the highest reliability among the traits discussed.Source: §4.1 · p. 8
• Competency ratings had lower reliability than personality traits, with modest absolute agreement but moderate consistency in candidate ranking.Consistency was especially evident for Development Orientation and overall interview performance.Source: §4.1 · p. 8

Construct validity

• Personality-targeted questions produced the strongest self-other agreement for Extraversion (r = 0.42) and Conscientiousness (r = 0.41).Agreeableness reached r = 0.25 and Honesty-Humility r = 0.22.Source: §4.2 · p. 9

Nomological association

• The competency regression model explained 87.1% of interview-performance variance (R2 = 0.871), with social versatility the strongest predictor (β = 0.405).All four competencies showed significant positive associations with interview performance.Source: §4.3 · pp. 9–10

Predictive validity

• Observer-rated personality models explained 17.7%–27.1% of competency and hireability variance, compared with 2.7%–11.0% for self-reported personality models.Generic- and personality-question observer models showed similar ranges.Source: §4.4 · p. 10

Fairness

• Personality-question observer ratings showed fewer significant demographic effects than generic-question observer ratings.Source: job related competencies · p. 11

Benchmark design

• Observer ratings from personality-targeted questions serve as benchmark labels because trait-relevant situations facilitate personality expression.The benchmark uses these ratings instead of generic-question ratings to elicit behavioral cues for AI assessment.Source: §5.1 · pp. 11–12
• The benchmark evaluates text-based, audio-based, visual-based, and multimodal methods, with subject-level splits and balanced demographic distributions.Training, validation, and test sets contain 70% (n = 452), 10% (n = 64), and 20% (n = 130) of subjects, respectively.Source: §5.1 · p. 12, §5.2 · p. 12

Evaluation

• MSE evaluates continuous-trait prediction, following prior AVI personality-assessment studies and benchmarks.MSE measures average squared differences between predicted and actual trait values.Source: §5.2 · p. 12

Text-based results

• PersonalityLLM achieved the best average results among text-based methods, including for H and A, whereas general-purpose GPT-4 and DeepSeek-R1 performed worse.The comparison indicates that task-specific personality alignment matters beyond model scale alone.Source: §5.3 · p. 12

Unimodal results

• Whisper, Emotion2Vec, and Wav2Vec2 outperformed handcrafted acoustic features, while Swin Transformer was the strongest visual baseline.Audio and visual-only methods generally produced higher MSEs than text-based methods, indicating more limited unimodal information.Source: §5.3 · p. 12

Multimodal results

• Multimodal methods achieved the best overall performance, but their advantage over strong text-based baselines was relatively small.EMMR achieved the best result for C, while PersonalityLLM achieved the best results for H and A; fusion effectiveness depended on integrating complementary cues.Source: §5.3 · p. 13

Trait coverage

• Personality-targeted questions cover four job-related HEXACO traits, while Emotionality and Openness rely on generic-question ratings and self-reports.Future studies should design and validate targeted questions for all HEXACO domains.Source: §6 · p. 14

Annotation

• Trained-rater BARS annotations may still reflect rater-specific preferences or demographic sensitivity.The paper calls for examining human rating decisions and whether trained models reproduce or amplify these patterns.Source: §6 · p. 14
• Observer-rated personality traits showed moderate to high consistency, but job-related competencies had lower absolute agreement and may require additional calibration.The limitation concerns competency judgments rather than the reported reliability of personality ratings.Source: §6 · p. 14

Benchmark scope

• The benchmark does not exhaust all modeling strategies, and multimodal fusion only slightly outperformed strong text-based baselines.Future work should investigate strategies that better integrate linguistic, acoustic, and visual cues.Source: §6 · p. 14

Dataset contribution

• AVI-Personality contains 3,876 videos from 646 participants answering generic and personality interview questions.The dataset includes multimodal behavior, self- and observer-rated personality, recruiter-rated competencies, interview performance, and cognitive ability.Source: §7 · p. 14

Validation

• Observer-rated personality traits, especially those based on personality-targeted questions, had reliable and meaningful annotations.The dataset was evaluated through reliability, construct validity, internal nomological association, and fairness analyses.Source: §7 · p. 14
• Job-related competencies significantly predicted interview performance.This result forms part of the dataset’s internal validation evidence.Source: §7 · p. 14

Benchmark conclusion

• Text-based methods provided strong personality-relevant signals, while multimodal methods achieved the best overall performance.The benchmark supports using AVI-Personality to develop and evaluate AI-based assessment algorithms.Source: §7 · p. 14