PaperlayerView exact revision on arXiv

Source-linked AI summary

Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization

Yeongtak Oh, Dongwook Lee, Sangkwon Park, Heeseung Kim, Sungroh Yoon

arXiv:2605.09996v2cs.CV

TL;DR

Omnimodal personalization lacks unified evaluation across text, image, and audio, especially for absent-persona cases and systematic grounding. Omni-Persona introduces a cross-modal benchmark and calibrated evaluation, revealing that visual grounding is more reliable than audio and that RLVR improves recall without calibrated abstention.

  • Problem

    Omnimodal personalization lacks unified text-image-audio benchmarks, absent-persona coverage, and systematic evaluation of realistic cross-modal grounding.

  • Method

    Omni-Persona models personalization as cross-modal routing over a Persona Modality Graph and benchmarks grounding across 4 task groups and 18 tasks.

  • Results

    Open-source models ground visual cues more reliably than audio cues; RLVR improves recall but fails to induce calibrated abstention, leaving calibrated accuracy below base.

  • Takeaways & Limitations

    Omni-Persona provides a diagnostic framework that exposes omnimodal personalization failure modes and guides future post-training and reward-design research.

  • Takeaways & Limitations

    The benchmark uses synthetic audio and text, while free-form LLM-as-a-judge evaluation may introduce residual bias.

Abstract

from arXiv · show

While multimodal large language models have advanced across text, image, and audio, personalization research has remained primarily vision-language, with unified omnimodal benchmarking that jointly covers text, image, and audio still limited, and lacking the methodological rigor to account for absent-persona scenarios or systematic grounding studies. We introduce Omni-Persona, the first comprehensive benchmark for omnimodal personalization. We formalize the task as cross-modal routing over the Persona Modality Graph, encompassing 4 task groups and 18 fine-grained tasks across ~750 items. To rigorously diagnose grounding behavior, we propose Calibrated Accuracy (Cal), which jointly evaluates correct grounding and appropriate abstention, incorporating absent-persona queries within a unified evaluation framework. On our dedicated experiments, three diagnostic findings emerge: (i) recent open-weight models show a consistent audio-vs-visual grounding gap that RLVR partially narrows via dense rule-based supervision; (ii) recall score and parameter scale are incomplete diagnostics, since strong recall can coexist with absent-persona hallucination and larger models do not always achieve higher Cal, exposing calibration as a separate evaluation axis; and (iii) SFT is limited by the scalability of ground-truth annotation. RLVR improves recall but not calibrated abstention under our reward design, leaving Cal at or below the base model. Omni-Persona thus serves as a diagnostic framework that surfaces the pitfalls of omnimodal personalization, guiding future post-training and reward design.

1 INTRODUCTION

Omni-Persona introduces a unified benchmark for omnimodal personalization that covers image, text, and audio, includes absent-persona cases, and evaluates both grounding and abstention. Its diagnostic analysis compares SFT and RLVR, revealing visual-over-audio grounding asymmetry and distinct post-training trade-offs.

  • Benchmark and formulation: Omni-Persona is an evaluation-only benchmark formalized by the Persona Modality Graph for cross-modal personalization with image, text, and audio contexts.It provides systematic cross-modal coverage, including audio as a persona modality and absent-persona cases.
  • Benchmark and formulation: The benchmark spans 4 task groups and 18 fine-grained tasks across ∼750 evaluation items, covering perceptual matching and grounded retrieval.Absent-persona samples place the ground-truth persona outside retrieved context, introducing retrieval noise.
  • Diagnostic analysis: The analysis systematically compares supervised fine-tuning and reinforcement learning with verifiable rewards to study post-training for omnimodal personalization.SFT is constrained by constructing high-quality, in-domain ground-truth supervision for diverse open-ended scenarios.
  • Evaluation methodology: Calibrated accuracy jointly evaluates correct grounding and appropriate abstention on absent-persona queries, exposing hallucination and over-abstention hidden by recall-only protocols.The evaluation couples unanswerable queries with hard distractors and retrieval noise.
  • Diagnostic analysis: Open-source models exhibit a visual-over-audio grounding asymmetry, while SFT and RLVR show distinct failure modes and trade-offs across tasks.These findings provide a model-specific diagnostic map for future omnimodal personalization research.

2 RELATED WORKS

Prior multimodal personalization methods evolved from brittle zero- or few-shot retrieval toward post-training approaches, while their evaluation protocols remain heavily reliant on recall-centric metrics. These metrics emphasize surface-level signals and underrepresent generation quality, absent-persona calibration, and post-training trade-offs.

  • Multimodal Personalization Methods: Early personalized VLMs use zero- or few-shot retrieval for user-defined concepts but remain brittle when new concepts must be incorporated dynamically into user memory.Post-training approaches subsequently emerged to mitigate this rigidity.
  • Evaluation Protocols for Multimodal Personalization: Existing evaluation protocols rely heavily on recall-centric metrics that reward surface-level signals such as name recall and contextual dialogue snippets.These signals can be directly reinforced during post-training.
  • Evaluation Protocols for Multimodal Personalization: Recall-centric evaluation underrepresents generation quality, calibration under absent-persona queries, and trade-offs introduced by post-training.The passage frames these as limitations of current evaluation protocols.

3 PROBLEM FORMULATION

Omni-Persona formulates omnimodal personalization as grounding responses in relevant visual, audio, and textual persona evidence, extending prior vision-language personalization with audio. The problem separates retrieval from contextual grounding and evaluates grounding under controlled post-retrieval conditions using shared persona contexts.

  • Formal Definition: Omnimodal personalization extends vision-language personalization by adding audio alongside visual and textual persona modalities.A user’s personal memory contains triplets of visual identity, audio sample, and text descriptor.
  • Formal Definition: Given multimodal query inputs, the model identifies the relevant memory entry while ignoring irrelevant modalities, then integrates its associated textual details into a grounded response.Retrieved memories are represented as aggregated top-K contexts, with each context containing visual, audio, and textual components.
  • Why Raw Context Matters: The formulation preserves raw images and audio because converting them into text can discard fine-grained identity cues such as vocal timbre and facial geometry.Raw omnimodal context is used instead of compact textual descriptions to avoid this information bottleneck.
  • Contextual Grounding over Retrieval: The work separates retrieval from contextual grounding and focuses on integrating retrieved multimodal evidence into faithfully personalized responses.Evaluation distinguishes grounding quality from retrieval quality by construction.
  • Task Composition and Controlled Post-retrieval Evaluation: Under post-retrieval evaluation, all models receive identical pre-retrieved persona contexts and must align queries with relevant visual, textual, or acoustic evidence.The contexts include visually similar identities and gender-matched voice samples as hard distractors.
  • Positioning within Retrieval-Augmented Personalization: The findings inform retrieval-augmented personalization by exposing generator limitations that retrieval improvements cannot fix and testing reliability under imperfect retrieval.Correctly retrieved evidence remains ineffective when the model cannot interpret or use it.

4 BENCHMARKING OMNIMODAL PERCEPTION AND RETRIEVAL

Omni-Persona formalizes omnimodal personalization as cross-modal routing over a Persona Modality Graph, benchmarking modality matching across text, image, and audio. Its design explicitly evaluates absent-persona abstention and robustness to noisy retrieval through hard distractors and no-GT contexts.

  • Persona Modality Graph and Task Formulation: Omni-Persona models personalization as cross-modal routing over a Persona Modality Graph whose nodes contain visual, audio, and textual modalities.The graph includes a query node and retrieved context nodes, with personalization scenarios defined by their interconnections.
  • Persona Modality Graph and Task Formulation: The benchmark covers four routing scenarios: Image-to-Image, Audio-to-Audio, Text-to-Text, and Text-to-Any matching.These represent visual identification, voice identification, same-modal semantic matching, and cross-modal semantic alignment, respectively.
  • Persona Modality Graph and Task Formulation: The framework supports absent-persona calibration by forming an active edge when the target is present and prompting abstention when it is absent.The formulation yields four scenario groups and 18 fine-grained tasks.
  • Benchmark Design Principles: Omni-Persona is the first personalization benchmark to treat audio as a full persona modality and unanswerable items as a primary evaluation dimension.Its cross-modal tasks also align audio evidence with visual descriptions, or visual evidence with audio descriptions, to expose modality-specific grounding biases.
  • Robustness Under Retrieval Imperfection: The benchmark tests retrieval robustness with hard distractors and no-GT retrieval, requiring structured abstention rather than hallucinated persona matching.Hard distractors use visually or vocally similar individuals, while no-GT retrieval omits the ground-truth persona from context.

5 EXPERIMENTS

The experiments benchmark post-training regimes for omnimodal personalization and diagnose grounding through recall, calibrated accuracy, and abstention. They find that RLVR improves answerable recall but does not reliably improve calibration, while scale and SFT expansion are insufficient guarantees of performance.

  • Experimental Setup: The study evaluates diverse omnimodal models and post-training regimes to identify behaviors and requirements for advancing personalization.It covers four open-source backbones under SFT-1K, SFT-10K, and RLVR, plus closed-source and open-source baselines.
  • Evaluation Metrics: Cal exposes grounding failures that recall alone hides by jointly evaluating answerable correctness and abstention on unanswerable items.Anti-hallucination metrics additionally report 1 −FA and TA, with higher values indicating better performance.
  • Model Comparisons: 35.7% Cal is achieved by Gemini-3.6-Flash despite 60.1% Ans, because low Unans reflects hallucination on absent-persona items.This demonstrates that strong answerable recall does not ensure reliable abstention.
  • Model Comparisons: 39.3% is MiniCPM’s highest open-source Cal, while Q3-30B scores 33.7% versus Q2.5-3B’s 36.7%, showing parameter scale alone does not guarantee calibration.Among the RLVR backbones, Q2.5-3B retains the highest Cal at 34.1%.
  • Post-Training Regimes: SFT expansion from 1K to 10K samples fails to yield consistent Cal improvements and degrades performance for the Gemma4 series.The authors attribute SFT limitations to noisy ground-truth synthesis, unpredictable test-time queries, and scarce large-scale paired real-world multimodal data.
  • Post-Training Regimes: Ans improves by +6.9 for Qwen 3B and +9.2 for Qwen 7B under RLVR, but these recall gains do not translate into calibration.RLVR improves recall on every backbone, with smaller gains of +2.8 and +1.8 for Gemma, while failing to induce abstention.

6 TOWARD OMNIMODAL PERSONALIZATION: IN-DEPTH ANALYSES

The analyses reveal a persistent audio–visual grounding gap and show that recall and model scale alone miss calibrated abstention failures. RLVR improves answerable recall but does not reliably produce calibrated abstention, while SFT is constrained by scalable supervision.

  • Cross-modal grounding: A 5–35 percentage-point answerable-recall gap separates A2A from I2I tasks, while naive RLVR improves audio recall but biases models away from abstention.Rule-based rewards improve A2A answerable recall across all four backbones.
  • Cross-modal grounding: Gemini models reach up to 40.0% A2A performance yet retain a clear cross-modal perceptual gap.Open-source models show substantially stronger visual than auditory perception.
  • Calibration: Recall and model scale are incomplete diagnostics because absent-persona hallucination and over-abstention can lower calibrated accuracy despite strong answerable performance.Cal jointly measures grounding on answerable cases and abstention on unanswerable cases.
  • Calibration: Gemini-3.6-Flash attains Ans 60.1% and TA (9.2%), capping calibrated accuracy at 35.7%.Q3-30B likewise has higher answerable recall but worse unanswerable-case performance than smaller Q2.5 variants, producing lower overall Cal.
  • Training methods: SFT is limited by scalable ground-truth annotation, whereas RLVR yields answerable-recall gains on every backbone but shifts models toward answering, with 1 −FA up and TA down.Scaling SFT data from 1K to 10K does not reliably improve Cal; RLVR instead uses outcome-level supervision.

7 CONCLUSION · A FURTHER RELATED WORKS · B PRELIMINARIES

Omni-Persona introduces a comprehensive omnimodal personalization benchmark centered on multimodal grounding, absent-persona evaluation, and calibrated anti-hallucination assessment. Its findings expose visual–audio reliability gaps, limits of scaling SFT, and recall improvements from RLVR that do not resolve calibration.

  • 7 CONCLUSION: Omni-Persona formalizes contextual grounding over retrieved persona evidence and raw omnimodal context, treating audio alongside images and text and absent-persona queries as core evaluation dimensions.The benchmark targets personalized expressiveness and systematic analysis of omnimodal personalization.
  • 7 CONCLUSION: Calibrated accuracy and anti-hallucination accuracy address recall-only metrics that can obscure hallucination under retrieval noise.These metrics extend evaluation beyond recall to account for hallucination behavior.
  • 7 CONCLUSION: Open-source models process visual cues substantially more reliably than audio cues on independently evaluated omnimodal tasks.The conclusion identifies a consistent modality-specific grounding gap in open-source models.
  • 7 CONCLUSION: Scaling SFT does not reliably improve performance, reflecting the difficulty of constructing data aligned with open-ended personalization.The limitation concerns supervision scalability for open-ended personalized behavior.
  • 7 CONCLUSION: The benchmark uses synthetic audio and text with rigorous model-based filtering, while human verification remains a future refinement and free-form LLM-as-a-judge evaluation may retain bias.Appendix H provides further discussion of residual evaluation bias.
  • A FURTHER RELATED WORKS: Recent text-only personalization benchmarks study profiling and user response but remain confined to text, lacking detailed multimodal-grounding analyses for omnimodal assistants.Examples include dynamic profiling, interaction-log alignment, and latent-trait inference from conversational history.
  • A FURTHER RELATED WORKS: Compared with CoViP, Omni-Persona adds audio personas, ground-truth-absent multi-distractor queries, and broader open-ended QA and cross-modal identity-matching tasks.The comparison positions Omni-Persona as broader in modalities, absent-persona evaluation, and task scope.
  • B PRELIMINARIES: Omnimodal foundation models target any-to-any generation across text, vision, and audio, while Gemma4 E-series models support constrained-memory, on-device-oriented analysis.Gemma4 uses Per-Layer Embeddings and Grouped-Query Attention to reduce KV-cache pressure, despite not being natively omnimodal.

C BENCHMARK DESIGN RATIONALE AND INTEGRITY … F.5 ADDITIONAL CROSS-JUDGE ROBUSTNESS

Omni-Persona is a rigorously constructed 750-item, 18-task benchmark spanning four omnimodal personalization scenarios, with absent-persona cases and modality controls designed to expose grounding and abstention behavior. Its analyses reveal recall–abstention trade-offs in supervision and rewards, complementary reward components, and robust calibrated-accuracy rankings across judges.

  • C BENCHMARK DESIGN RATIONALE AND INTEGRITY: Approximately 750 instances span 18 fine-grained tasks, with synthetic training images, disjoint audio splits, manual verification, and modality-appropriate distractors preventing contamination and shortcuts.Absent-persona cases comprise approximately half the benchmark and test abstention when target evidence is unavailable.
  • E EVALUATION DATASET CONFIGURATIONS: Removing both image and audio reduces I2I recall from 28 to 3 for Qwen2.5-Omni-3B and from 44 to 3 for Gemma4-E2B, while targeted corruption selectively harms the corresponding perceptual task.T2T and T2Any improve after perceptual inputs are removed, confirming modality-relevant dependence rather than a shared textual shortcut.
  • D DETAILS ON EVALUATION DATASET AND METRICS: The four PMG groups comprise I2I and A2A identity routing, T2T semantic matching, and the harder T2Any text-to-audio-to-target regime, with no-GT variants for every sub-task.The benchmark uses highly similar distractors and dense interleaved image-audio-text contexts to require fine-grained cross-modal discrimination.
  • F.1 SUPERVISION-INDUCED RECALL–ABSTENTION TRADE-OFF: Answerable recall declines from 43.0 at a 1:0 training ratio to 24.3 at approximately 1:1, while true-abstention accuracy rises from 0.6 to 63.0.The 1:0 setting maximizes recall but almost never abstains; 1:1 maximizes aggregate calibration while reducing recall below the base model by 9.7 points, whereas 1:0.6 is near-base recall with substantial abstention.
  • F.2 STATISTICAL SIGNIFICANCE OF THE RECALL–CALIBRATION TRADE-OFF: RLVR significantly improves Ans and 1 −FA while significantly reducing TA on both Qwen2.5-Omni backbones, producing a modest, nonsignificant decrease in Cal.Paired McNemar testing with 95% confidence intervals supports a shift toward answering rather than improved calibration.
  • F.4 REWARD-COMPONENT ABLATION: Removing localization reduces overall recall from 40.7% to 31.5% while raising TA from 26.7% to 61.3%, whereas removing verification lowers Cal to 35.3% versus 36.7% for the base model.The full reward yields the strongest overall and perceptual recall; text-QA-only supervision underperforms the base model.
  • F.5 ADDITIONAL CROSS-JUDGE ROBUSTNESS: GPT-5.6-sol agrees with the primary judge on 80.6–92.1% of items, with calibrated-accuracy rank correlation ρ = 0.91 (p < 0.001) versus ρ = 0.66 (p = 0.026) for answerable recall.Alternative judges shift absolute answerable scores more than calibrated-accuracy rankings, which remain reasonably robust.

F.6 ALIGNMENT OF EVALUATION METRICS · G ADDITIONAL EXPERIMENTAL CONFIGURATIONS · G.1 DETAILS ON SFT IMPLEMENTATIONS

The evaluation analysis separates answerable quality from unanswerable abstention because lexical metrics do not reliably measure both. Additional configurations show that RLVR improves lexical overlap and judge alignment, while SFT uses frozen encoders, LoRA, and multimodal data designed to teach grounding and missing-context recognition.

  • F.6 ALIGNMENT OF EVALUATION METRICS: Answerable recall increases with ROUGE-L, but unanswerable recall shows little association, so abstention must be evaluated separately.Content-overlap metrics primarily explain answerable-query performance and fail to capture abstention on unanswerable queries.
  • F.6 ALIGNMENT OF EVALUATION METRICS: RLVR raises ROUGE-L by +9.2/+10.0 points on the Qwen backbones and strengthens agreement between the LLM judge and lexical metrics.RLVR shifts outputs toward answering, unlike SFT, which largely preserves the base policy’s lexical profile.
  • F.6 ALIGNMENT OF EVALUATION METRICS: ROUGE-L agreement with LLM-as-a-judge evaluations is stronger than token-level F1 on answerable items across all models.The benchmark reports answerable QA quality and unanswerable abstention behavior separately because judge and lexical or semantic metrics capture distinct signals.
  • F.6 ALIGNMENT OF EVALUATION METRICS: RL-tuned models attain mean ρ = 0.70 versus 0.63 for base policies, yielding the strongest post-training alignment between judge and lexical metrics.Strong closed-source and base models also maintain high concordance on answerable items.
  • G.1 DETAILS ON SFT IMPLEMENTATIONS: SFT minimizes standard autoregressive negative log-likelihood on triplets (q, C, y∗), pairing queries and omnimodal contexts with ground-truth responses.The training corpus is DSFT, with q as the query, C as the omnimodal context, and y∗ as the ground-truth response.
  • G.1 DETAILS ON SFT IMPLEMENTATIONS: SFT fine-tunes Qwen2.5-Omni (3B/7B) and Gemma4 (E2B/E4B) for 3 epochs at lr = 2 × 10−5 with frozen vision/audio encoders.Qwen uses LoRA with r = 64, α = 128, while Gemma4 uses r = 32, α = 64.
  • G.1 DETAILS ON SFT IMPLEMENTATIONS: The 10K-scale SFT corpus covers 12 task types spanning grounding, audio scenarios, and missing-modality variants that train recognition of absent context.The corpus also applies augmentation to mitigate positional and cross-modal shortcut biases.
  • G.1 DETAILS ON SFT IMPLEMENTATIONS: The pipeline uses 1K and 10K SFT corpora for modality-alignment scaling, followed by RLVR data targeting audio/image perception and text-grounded retrieval.Across SFT and RLVR, absent-persona (no-GT) samples account for approximately 37.

G.2 DETAILS ON RLVR IMPLEMENTATIONS

The RLVR implementation uses GSPO with KL regularization and task-dispatched, error-softened rewards for perception, localization, and retrieval. Despite explicitly rewarding abstention, the resulting policies remain biased toward answering.

  • RLVR Objective: GSPO optimizes verifiable reward while penalizing KL divergence from a reference policy, using β = 0.04.The objective maximizes E[r(y, q, C) − βD_KL(πθ∥πref)].
  • RLVR Configurations: RLVR trains without SFT warmup on 1K samples, with G = 4 generations, lr = 1 × 10−5, and LoRA r = 32, α = 64.All perceptual encoders remain frozen, and hyperparameters were tuned primarily on Qwen2.5-Omni-3B before reuse across backbones.
  • Verifiable Reward Designs: Retrieval VR evaluates grounded answers when the persona is present and abstention when absent, but resulting policies still exhibit a bias toward answering.Correct answers and abstentions receive +1.0, while false negatives and false positives receive −0.5.
  • On-Policy RLVR Algorithms: GSPO is chosen because the verifier produces sequence-level rewards from rule-based verification and LLM judging rather than token-level supervision.GSPO applies sequence-level importance-ratio clipping while sharing the KL reference with token-level alternatives.

H LIMITATIONS AND BROADER IMPACTS

The benchmark relies on synthetic training personas and broad-coverage SFT analysis because realistic, evaluation-aligned data are difficult to scale without privacy, consent, licensing, or contamination risks. Its evaluation and reward design remain limited by lexical abstention detection, restricted perceptual uncertainty coverage, synthetic demographic bias, and post-retrieval scope.

  • Data and Training Limitations: Training uses synthetic personas while reserving real images for evaluation, creating a deliberate synthetic–real domain split driven by privacy, consent, and licensing constraints.The training personas use TTS-generated voice clips and generated facial images; real images are reserved strictly for the Omni-Persona evaluation benchmark.
  • Data and Training Limitations: Evaluation-aligned in-domain SFT data are difficult to construct at scale because high-quality answers risk test-set exposure and benchmark contamination.The authors therefore treat SFT results as an analysis of broad-coverage training.
  • Reward and Evaluation Limitations: Fine-grained reward-shaping ablations for balancing grounding and abstention remain future work, despite stability filters that prevent severe RL degeneration.The RLVR pipeline includes 4-gram repetition, character-diversity, and sentence-repetition checks.
  • Reward and Evaluation Limitations: Abstention is detected through predefined lexical phrases during both RLVR training and evaluation, avoiding an additional judge model but limiting the detection design.Applying the same rule in training and evaluation makes abstention assessment directly comparable.
  • Benchmark Scope: The benchmark tests fine-grained visual and auditory discrimination in provided query–context settings but does not fully represent open-ended uncertainty in unconstrained real-world environments.Included difficulties include background speech, conversational voices, perceptually similar distractors, and missing textual information.
  • Broader Impacts: Synthetic persona generation may underrepresent demographic groups, while fairness across multilingual voice, facial, and linguistic attributes remains an important future evaluation direction.The benchmark includes hard distractors and retrieval noise to test robustness, but fairness coverage remains incomplete.
  • Future Directions: Future work should extend evaluation beyond post-retrieval grounding and generation to end-to-end retrieval-augmented pipelines that separate retriever errors from downstream grounding failures.The framework could also extend to on-device personalization benchmarks built from unstructured omnimodal data.

I FURTHER ANALYSIS ON LEXICAL METRICS

Lexical metrics broadly validate LLM-judge verdicts, with CORRECT predictions scoring substantially higher than WRONG ones on ROUGE-L. SFT largely preserves base-model behavior and does not reliably reshape abstention boundaries.

  • Lexical Metrics Validate, but Cannot Replace, the LLM Judge: Across evaluated backbones, judge-labeled CORRECT predictions consistently achieve substantially higher ROUGE-L than WRONG predictions.The analysis partitions answerable predictions (N = 391) by judge verdict before recomputing lexical scores.
  • Lexical Metrics Validate, but Cannot Replace, the LLM Judge: 2–4× higher ROUGE-L typically separates predictions judged CORRECT from those judged WRONG.This recurring gap supports consistency between LLM-judge verdicts and reference-based lexical signals.
  • Post-training Regimes Show Different Behavioral Fingerprints: SFT-10K largely preserves base-model behavior, with only marginal ROUGE-L changes and nearly unchanged genuine hallucination rates.Broader supervised data alone therefore does not reliably reshape the abstention boundary in open-ended personalization settings.

J DETAILED TASK TAXONOMY AND GRANULARITIES · K USED TEMPLATES FOR DATASET CONSTRUCTION

The benchmark taxonomy organizes Omni-Persona into 4 matching scenario groups and 18 modality-target sub-tasks, each with an absent-persona variant requiring structured abstention. Dataset construction and evaluation use fixed prompts for benchmarking, answer judging, and automated persona profiling.

  • J DETAILED TASK TAXONOMY AND GRANULARITIES: 4 matching scenario groups and 18 sub-tasks are derived from the Persona Modality Graph formulation.Groups are defined by query and context modalities, while sub-tasks sweep the retrieval target modality.
  • J DETAILED TASK TAXONOMY AND GRANULARITIES: Each sub-task includes an absent-persona variant in which the target is absent and the correct behavior is structured abstention.The variant is defined as no-GT, eq→j=0, target=None.
  • J DETAILED TASK TAXONOMY AND GRANULARITIES: The taxonomy spans query, context, and retrieval modalities represented by image, text, and audio or voice samples.The notation uses I for image, T for text, and Av for audio/voice sample.
  • J DETAILED TASK TAXONOMY AND GRANULARITIES: Illustrative benchmark examples visualize only answerable instances where the target persona is present.Absent-persona cases are therefore not represented in the figure’s examples.
  • K USED TEMPLATES FOR DATASET CONSTRUCTION: A fixed prompt template is used for evaluation in the Omni-Persona benchmark.The cited template standardizes the evaluation prompt across benchmark instances.
  • K USED TEMPLATES FOR DATASET CONSTRUCTION: A judge prompt is used to assess answer correctness.This provides a separate prompt for correctness judging in the evaluation pipeline.
  • K USED TEMPLATES FOR DATASET CONSTRUCTION: A structured prompt supports automated persona profiling and attribute enrichment.The prompt is used to visualize the structured profiling input for dataset construction.
Loading 2605.09996v2…