PaperlayerView exact revision on arXiv

Source-linked AI summary

MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook

Peng Xu, Shengwu Xiong, Jiajun Zhang, Yaxiong Chen, Bowen Zhou, Chen Change Loy, David A. Clifton, Kyoung Mu Lee, Luc Van Gool, Ruiming He, Ruilin Yao, Xinwei Long, Jirui Huang, Kai Tian, Sa Yang, Yihua Shao, Jin Feng, Yue Zhong, Jiakai Zhou, Cheng Tang, Tianyu Zou, Yifang Zhang, Junming Liang, Guoyou Li, Zhaoxiang Wang, Qiang Zhou, Yichen Zhao, Shili Xiong, Hyeongjin Nam, Jaerin Lee, Jaeyoung Chung, JoonKyu Park, Junghun Oh, Kanggeon Lee, Wooseok Lee, Juneyoung Ro, Turghun Osman, Can Hu, Chaoyang Liao, Cheng Chen, Chengcheng Han, Chenhao Qiu, Chong Peng, Cong Xu, Dailin Li, Feiyu Wang, Feng Gao, Guibo Zhu, Guopeng Tang, Haibo Lu, Han Fang, Han Qi, Hanxiao Wu, Haobo Cheng, Hongbo Sun, Hongyao Chen, Huayong Hu, Hui Li, Jiaheng Ma, Jiang Yu, Jianing Wang, Jie Yang, Jing He, Jinglin Zhou, Jingxuan Li, Josef Kittler, Lihao Zheng, Linnan Zhao, Mengxi Jia, Muyang Yan, Nguyen Thanh Thien, Pu Luo, Qi Li, Shien Song, Shijie Dong, Shuai Shao, Shutao Li, Taofeng Xue, Tianyang Xu, Tianyi Gao, Tingting Li, Wei Zhang, Weiyang Su, Xiaodong Dong, Xiao-Jun Wu, Xiaopeng Zhou, Xin Chen, Xin Wei, Xinyi You, Xudong Kang, Xujie Zhou, Xusheng Liu, Yanan Wang, Yanbin Huang, Yang Liu, Yang Yang, Yanglin Deng, Yashu Kang, Ye Yuan, Yi Wen, Yicen Tian, Yilin Tao, Yin Tang, Yipeng Lin, Yiqing Wang, Yiting Xi, Yongkang Yu, Yumei Li, Yuxin Qin, Yuying Chen, Yuzhe Cen, Zhaofan Zou, Zhaohong Liu, Zhehao Shen, Zhenglin Du, Zhengyang Li, Zhenni Huang, Zhenwei Shao, Zhilong Song, Zhiyong Feng, Zhiyu Wang, Zhou Yu, Ziang Li, Zihan Zhai, Zijian Zhang, Ziyang Peng, Ziyun Xiao, Zongshu Li

arXiv:2509.14142v1cs.CV

TL;DR

Synergistic effects among reasoning tasks and non-stepwise complex reasoning remain understudied in real-world and specialized multimodal settings. MARS2 examines these issues through two tailored datasets and three competition tracks; 76 teams submitted 1200+ entries, with 40+ submissions included in the final rankings.

  • Problem

    Synergistic effects among reasoning tasks and non-stepwise complex reasoning are understudied challenges in real-world and specialized multimodal scenarios.

    Source: §2.2 · p. 4, §5 · p. 17

  • Method

    The challenge releases Lens and AdsQA for general reasoning in 12 daily scenarios and advertisement-video reasoning, and defines three open-ended QA tracks.

    Source: §2.2 · p. 5, §5 · p. 17

  • Results

    76 teams registered and submitted 1200+ entries, with 40+ submissions included in the final ranking lists.

    Source: §5 · p. 17

  • Takeaways & Limitations

    Lens’s shared samples support studying reasoning-task synergy, while AdsQA supports studying non-stepwise complex reasoning in advertising.

    Source: §2.2 · p. 4

    Current MLLMs often confuse similar materials or struggle to identify semantic targets in fine-grained images.

    Source: §4.1 · p. 15

Abstract

from arXiv · show

This paper reviews the MARS2 2025 Challenge on Multimodal Reasoning. We aim to bring together different approaches in multimodal machine learning and LLMs via a large benchmark. We hope it better allows researchers to follow the state-of-the-art in this very dynamic area. Meanwhile, a growing number of testbeds have boosted the evolution of general-purpose large language models. Thus, this year's MARS2 focuses on real-world and specialized scenarios to broaden the multimodal reasoning applications of MLLMs. Our organizing team released two tailored datasets Lens and AdsQA as test sets, which support general reasoning in 12 daily scenarios and domain-specific reasoning in advertisement videos, respectively. We evaluated 40+ baselines that include both generalist MLLMs and task-specific models, and opened up three competition tracks, i.e., Visual Grounding in Real-world Scenarios (VG-RS), Visual Question Answering with Spatial Awareness (VQA-SA), and Visual Reasoning in Creative Advertisement Videos (VR-Ads). Finally, 76 teams from the renowned academic and industrial institutions have registered and 40+ valid submissions (out of 1200+) have been included in our ranking lists. Our datasets, code sets (40+ baselines and 15+ participants' methods), and rankings are publicly available on the MARS2 workshop website and our GitHub organization page https://github.com/mars2workshop/, where our updates and announcements of upcoming events will be continuously provided.

1. Introduction

MARS2 argues that multimodal research should move beyond familiar tasks toward challenging, specialized scenarios requiring complex reasoning and slower “System 2” thinking. Its 2025 challenge introduced tailored datasets and tracks, evaluated diverse models, and publicly released resources to support research and reproducibility.

Motivation and objectives

  • MARS2 targets challenging, specialized multimodal tasks requiring complex reasoning beyond stepwise problems, including evaluation of synergistic reasoning across tasks.The organizers identify a gap in benchmarks that do not ensure tasks share a data distribution, limiting evaluation of task synergy.Source: §1 · p. 2
  • The challenge released Lens and AdsQA as test sets to draw attention to synergistic reasoning across tasks and non-stepwise reasoning.The workshop aims to bring together perspectives and help researchers follow progress in multimodal machine learning and multimodal large language models.Source: §1 · p. 2

Tailored datasets

  • Lens supports general reasoning in 12 daily scenarios, while AdsQA targets domain-specific reasoning in advertisement videos; Lens samples support both VG-RS and VQA-SA.Lens samples were manually collected from social media, with 53% posted after January 2025; AdsQA is described as the first ad-video test set for tasks evaluating LLMs.Source: §1 · pp. 2–3

Challenge overview

  • The challenge offered three tracks: Visual Grounding in Real-world Scenarios (VG-RS), Visual Question Answering with Spatial Awareness (VQA-SA), and Visual Reasoning in Creative Advertisement Videos (VR-Ads).It ran in two stages for more than two months; 76 teams registered, submitted 1200+ entries, and 40+ valid submissions entered the final rankings.Source: §1 · pp. 2–3

Benchmark and reproducibility

  • The benchmark evaluated 40+ generalist and task-specific baselines and included 40+ valid participant solutions, spanning open-source and commercial models.The report describes model sizes from 3B to 72B and comparisons across generalist versus specialist, large versus small, and open-source versus closed-source models; datasets, code, and rankings were made public, with post-competition discussions and reopened submissions for ablations.Source: §1 · p. 3

2. Challenge

The challenge introduces Lens for multi-level reasoning across everyday images and AdsQA for specialized reasoning in advertisement videos, organizing evaluation into three tracks with distinct task and scoring protocols.

2.1.1. Lens Dataset

  • Lens [142] has 3.4K images and 60K+ questions across three tiers, eight tasks, and 12 daily scenarios.Rich annotations support varied prompts and downstream tasks; 53% of images were published after January 2025, helping limit the influence of models’ inherent knowledge.Source: §2.1.1 · p. 3
  • Lens reasoning-task accuracy remained below 60% across evaluated frontier MLLMs, with Qwen2.5-VL-32B at 48.47% Acc.@0.5 [142].The paper attributes the visual-grounding difficulty to complex queries, varied image resolutions, and numerous small ground-truth boxes.Source: §2.1.1 · p. 3

2.1.2. AdsQA Dataset

  • AdsQA [84] contains 10,962 clips from 1,544 advertisement videos and tests five open-ended QA skills, with a 555-video, 3,000+ question test set.Its clue-rich, information-dense ads require reasoning about implicit, non-physical, and mental content beyond conventional physical-scene perception and stepwise “if A, then B” reasoning; it is presented as the first advertisement-domain video QA benchmark for LLMs.Source: §2.1.2 · pp. 3–4
  • On AdsQA, Gemini 2.5 Pro achieved 60.7% accuracy, while five non-expert evaluators averaged 71.4% on 200 QA pairs [84].Models can miss subtle visual clues and fail to connect them to an advertisement’s theme and creative intent.Source: §2.1.2 · p. 4

2.2. Competition Tracks

  • The three open-ended tracks use shared Lens samples for VG-RS and VQA-SA, while VR-Ads uses AdsQA to probe implicit advertisement reasoning.VG-RS targets scene perception, localization, and spatial reasoning; VQA-SA tests spatial, commonsense, and counterfactual reasoning; VR-Ads targets implicit, non-physical, and abstract concepts.Source: §2.2 · pp. 4–5

2.3. Evaluation Protocols

  • Evaluation uses Acc.@0.5 for VG-RS, GLM4-flash2 majority-vote accuracy for VQA-SA, and gpt-4o-2024-08-06 rubric-scored average similarity for VR-Ads.For VG-RS, a box is correct when IoU exceeds 0.5; VR-Ads scores answers 1.0, 0.5, or 0.0 based on matching ground truth while excluding unsupported content.Source: §2.3 · p. 5

2.4. Terms, Phases, and Other Details

  • The leaderboard ran in two stages, receiving about 900 submissions in the public stage and about 300 in the final stage, totaling over 1,200.Participants received immediate EvalAI feedback; the committee imposed no model-size restrictions and allowed open-source and commercial models.Source: §2.4 · p. 5

3. Methods and Results

This section reports challenging benchmark results and describes leading methods for spatial reasoning and creative-advertisement video analysis, including synthetic or augmented data, staged training, fine-tuning, structured prompts, multi-query pipelines, and collaboration between generalist and specialist models. Reported results include Qwen2.5-VL’s accuracy advantage in TFA and a distilled 7B model matching its 72B teacher’s VQA-SA precision.

3. Methods and Results

  • VG-RS winners scored below 70%, while Track #3 peaked at 56% accuracy versus human performance of ∼70% [84], underscoring difficulty on Lens and AdsQA.The paper identifies these results as evidence that reasoning in complex scenarios and specialized domains remains challenging.Source: §3 · p. 7

3.1.1. VG-SMART

  • VG-SMART won Track #1 with 0.6671 by combining SNR-filtered data synthesis, grounding SFT, IoU-focused RL, and distillation from a 72B model to 7B.The synthesis process ensembles models to produce high-confidence, diverse training samples; SFT adapts the model to the track, and RL focuses on the IoU=0.5 boundary.Source: §3.1.1 · p. 8

3.1.2. DCM-VG

  • DCM-VG uses Qwen2.5-VL to propose boxes and Grounding DINO to verify and refine them, earning second place in Track #1 with 0.65.Its strategy also uses an enhanced prompt for small or obscured targets and multi-resolution inference to address scale sensitivity.Source: §3.1.2 · p. 9

3.1.3. VR-VG

  • VR-VG synthesizes 2.7 million QA pairs and applies SFT followed by GRPO with format and dense-IoU rewards, whose combination adds +3.2 mIoU.Its five-step data pipeline runs from global captioning and entity extraction through grounding, region refinement, and question synthesis.Source: §3.1.3 · p. 9

3.1.4. RVG-CVME

  • FVG-CVME is a reinforced visual-spatial grounding model integrating candidate voting and model ensembling; it trains a Qwen2.5-VL-3B base model with SFT and GRPO before training-free ensembling.SFT uses HumanRef-CoT, and aggregation uses VLM-R1-3B, Qwen2.5-VL-32B, and Qwen2.5-VL-72B with weighted box fusion.Source: §3.1.4 · pp. 9–10

3.1.5. KEME; 3.1.6. A VLM-R1-based Visual Grounding Method

  • KEME preprocesses images and bilingual queries, adds Qwen-generated scene context grounded with Florence-2, and selects detector boxes; CV and RL’s VLM-R1 method uses GRPO rewards and placed eighth.KEME falls back to a deterministic strategy if its evaluator finds no suitable detector candidate; CV and RL assigns reward 1 at IoU ≥0.5 and 0 otherwise.Source: §3.1.5 · p. 10, §3.1.6 · p. 10

3.2.1. RSVT

  • RSVT builds a roughly 170k-sample bilingual spatial-reasoning set, augments it with self-consistency and viewpoint transformations, and uses three-model majority voting.The champion solution trains three MLLMs and applies perspective-consistent handling for multi-turn questions.Source: §3.2.1 · pp. 10–11

3.2.2. SMART

  • SMART combines a synthesis, adaptation, reasoning, and testing framework with a two-stage training pipeline, placing second in its final evaluation with accuracy 0.7261.Its synthetic spatial QA data comes from rendered scenes with recorded object geometry and templated questions across three difficulty levels.Source: §3.2.2 · p. 11
  • SMART fine-tunes with location-aware synthetic QA and cold-start training followed by DAPO reinforcement learning, then selects among reasoning trajectories with VisualPRM-8B.At inference, GPT-4o refines context-dependent questions and image resolution is enhanced to retain fine-grained spatial cues.Source: §3.2.2 · pp. 11–12

3.2.3. STAGES

  • The Qwen2.5-VL 7B student reaches 69.72% VQA-SA precision, matching its 72B teacher.STAGES combines prompt optimization, large-scale SFT on synthetic data, in-domain SFT on high-confidence samples, and knowledge distillation; shared-image question prompts improved multiple models by 2–7 percentage points.Source: §3.2.3 · p. 12

3.2.4. o3-MQP

  • o3-MQP groups same-image questions by difficulty and batches them into one prompt, using shared context and tools such as bounding-box annotation, cropping, and object classification.Simpler questions supply context for more complex ones, supporting consistency across the multi-query answers.Source: §3.2.4 · p. 12

3.2.5. CCP4SR

  • CCP4SR addresses multi-turn spatial ambiguity by preserving image-level question order for coreference and causal links, and adding depth cues for direction questions.Its depth-informed approach helps with basic depth-related questions, but overall performance remains slightly inferior.Source: §3.2.5 · pp. 12–13

3.2.6. Prompt4SA

  • Prompt4SA uses five-stage cognition-guided prompting to resolve objects and reference frames before computing spatial relations, achieving high and stable Track #2 accuracy without task-specific fine-tuning.The stages classify question complexity, bind features, set the reference frame, compute relations, and verify and format the answer.Source: §3.2.6 · p. 13

3.3.1. Hi-CoT

  • Hi-CoT guides ad-video reasoning from audio-visual global understanding through shot-level alignment and hierarchical causal and commonsense inference to answer generation.Its process transcribes audio, analyzes scene-segmented shots, and connects observed evidence to underlying reasons.Source: §3.3.1 · pp. 13–14

3.3.2. TFA

  • Qwen2.5-VL scores 0.55 (72B), 0.51 (32B), and 0.46 (7B), versus GPT-4o at 0.50 and Qwen2.5-Omni at 0.44 (7B).The team reports that larger Qwen2.5-VL models improve video understanding and reasoning, while Qwen2.5-Omni audio yields no measurable improvement, likely because ad audio is mostly background music.Source: §3.3.2 · p. 14

3.3.3. T-STAR

  • T-STAR’s top-performing submission uses 2 FPS with maximum frame resolution, favoring spatial detail over higher temporal sampling for abstract ad reasoning.Its inference strategy tunes frame rate, resolution, and rejection sampling that discards answers longer than 30 words.Source: §3.3.3 · p. 14

4. Discussion and Outlook

The discussion finds that strong LLM-based submissions still face reliability, generalization, and reasoning failures in real-world and specialized settings. The outlook calls for broader scenarios, better testbeds, and expanded challenge tracks.

4.1. Further Discussion

  • Submitted methods gain performance from strong LLMs and techniques, but remain limited in reliability and generalization; optimizing only IoU may undermine other capabilities.Examples use task-oriented RL with IoU as the reward signal.Source: §4.1 · p. 15
  • Winning methods struggle with fine-grained objects and semantic targets, while perspective biases and misunderstandings of distance reveal weak physical commonsense in real-world scenes.Examples include confusing plastic bags with other objects and failing to identify a bookshelf filled with books.Source: §4.1 · pp. 15–16, Figure 4 · p. 16
  • On AdsQA, Gemini 2.5 Pro can give suboptimal answers, hallucinate content, omit advertisement details, and incorporate inherent biases.The paper presents three randomly selected failure cases for this state-of-the-art model.Source: §4.1 · p. 16, Figure 5 · p. 16

4.2. Outlook

  • The outlook values both generalist and specialist models and calls for more real-world scenarios, high-quality testbeds, finer-grained advertisement reasoning, more tracks, and larger test sets.Planned advertisement-video analysis includes persuasive strategies, core themes, and emotional appeals.Source: §4.2 · pp. 16–17

5. Conclusion

MARS2 2025 targets complex multimodal reasoning in real-world and specialized settings through tailored datasets and competition tracks. Its challenge, evaluated baselines, and participant solutions form a broad benchmark, with resources released publicly and further scenario development planned.

5. Conclusion

  • Lens and AdsQA support general reasoning across 12 daily scenarios and domain-specific reasoning in advertisement videos, respectively, across three tracks: VG-RS, VQA-SA, and VR-Ads.The challenge focuses on synergistic effects among reasoning tasks and non-stepwise complex reasoning.Source: §5 · p. 17
  • 76 teams registered and submitted 1200+ entries, with 40+ submissions included in the final ranking lists.The challenge lasted more than two months and drew teams from academic and industrial institutions.Source: §5 · p. 17
  • 40+ baselines, spanning generalist MLLMs and task-specific models, and participant models ranging from 3B to 72B form a benchmark comparing model types and scales.The comparisons cover generalist versus specialist, large versus small, and open-source versus closed-source models.Source: §5 · p. 17
  • Submitted solutions commonly use ensembles, data augmentation, prompt engineering, and alignment training, while many teams pair state-of-the-art MLLM base models with multi-step SFT and RL alignment.Collaboration between generalist and specialist models is widely used and achieves good performance in the challenges.Source: §5 · p. 17
  • Lens, AdsQA, code sets for 40+ baselines and 15+ participant methods, and rankings are publicly available, with future work committed to new scenarios, high-quality data, and open-source community growth.The report frames MARS2 as an effort to advance challenging, specialized tasks requiring complex multimodal reasoning and “System 2” slow thinking.Source: §5 · p. 17

A. Related Work · A.1. Multimodal Reasoning with Large Vision-Language Models.

MLLM reasoning remains exploratory despite strong recent models: they struggle with spatial, compositional, and abstract reasoning, while existing approaches show particular limits on abstract advertisement-video tasks. MARS2 addresses fragmented evaluation with a unified framework spanning three complementary tracks.

A.1. Multimodal Reasoning with Large Vision-Language Models.

  • Multimodal chain-of-thought methods such as Multimodal-CoT [159], LLaVA-CoT [130], and RAVEN-CoT [169] improve accuracy on ScienceQA and OK-VQA [90] and control reasoning trajectories in VG-RS and VQA-SA.These methods explicitly construct reasoning steps within visual prompts, but rely heavily on explicit visual cues and can produce vague chains for advertising metaphors, emotional undertones, and abstract semantics.Source: §A.1 · p. 24
  • RL-based methods Visual-RFT [80], SRPO [118], and VL-Rethinker [120] perform strongly on spatial, multi-step VQA-SA reasoning, but explicit rewards and high-quality feedback constrain generalization to abstract VR-Ads reasoning.The approaches refine reasoning trajectories using visual and textual reward signals, group-wise relative policy optimization, and self-reflection or soft replay.Source: §A.1 · p. 24
  • Native architectures including Gemini [116], GPT-4V [138], and Claude offer end-to-end multimodal reasoning, while open-source LLaVA [75] and MiniGPT-4 [167] show competitive visual grounding and spatial reasoning.Unified model designs and multimodal pretraining support cross-modal fusion, but these architectures still face challenges in abstract VR-Ads reasoning.Source: §A.1 · p. 24
  • Despite marked progress in text-only LLM reasoning, extending those capabilities to MLLMs remains in an early exploratory phase because of modality heterogeneity and cross-modal reasoning complexity.The related work groups current research into multimodal chain-of-thought, reinforcement-learning-based reasoning, and native multimodal architectures.Source: §A.1 · p. 24
  • Lens shows that GPT-4o [1], Gemini 2.5 Pro [116], InternVL3 [168], and Qwen2.5-VL [8] underperform in spatial, compositional, and abstract reasoning, especially hierarchical tasks.The paper identifies limitations in fundamental visual perception and logical reasoning.Source: §A.1 · p. 24
  • MARS2 addresses fragmented evaluation with a unified framework spanning visual grounding in physical scenes (VG-RS), spatially aware VQA (VQA-SA), and creative-advertisement video reasoning (VR-Ads).The framework is intended to provide a more comprehensive benchmark platform for advancing MLLM reasoning.Source: §A.1 · p. 24

A.2. Visual Grounding and Scene Perception

The section surveys visual grounding methods across external visual encoding, cross-modal semantic alignment, and multimodal reasoning, then presents Lens as a real-world benchmark targeting spatial reasoning that existing evaluations often miss.

Integration of external visual information

  • External visual encoders support grounding by supplying region-level, multi-scale, or mask-based image representations to LLM-based systems.Groma uses DINO and region tokens [87] [153]; Ferret and Ferret-v2 use multi-scale visual sampling [155]; Groundhog and GLaMM support mask generation.Source: §A.2 · p. 24

Cross-modal semantic alignment

  • Cross-modal alignment methods use learnable components to connect vision and language and support more adaptable localization and multimodal interaction.BLIP-2 and Grounding-GPT use Q-Former fusion [68] [73], LION uses multi-granularity adapters and a router [16], and Llava-grounding and Grounding-GPT support mixed box, text, and mask inputs and outputs [73].Source: §A.2 · p. 25

Multimodal reasoning strategies

  • Multimodal reasoning approaches extend grounding with region-aware reasoning and staged instruction tuning to improve reference resolution and cross-modal alignment.Shikra introduces Grounding CoT [19], while Kosmos-2 uses a three-stage perception, instruction-tuning, and alignment paradigm [96]. VPP-LLaVA uses a roughly 600k-sample grounding dataset for multi-instruction tuning [114].Source: §A.2 · p. 25
  • Later grounding work extends multi-instruction training to text-rich images, video, multiple modalities, and expanding task settings.TRIG covers forms, documents, and receipts, including localized answer alignment for Doc-VQA [71]; VideoGLaMM and LLaVA-ST address spatiotemporal or unified temporal-spatial localization [67]. VistaLLM trains across images, video, and audio, while LLaVA-c aims to mitigate catastrophic forgetting as tasks expand [78].Source: §A.2 · p. 25

Summary

  • Existing visual-grounding benchmarks range from referring-expression datasets to scene-graph, region-grounding, compositional-alignment, and MLLM generalization evaluations.The section notes that established datasets such as RefCOCO, RefCOCO+, RefCOCOg, ReferItGame, and RefCLEF do not meet MLLM reasoning evaluation needs; GQA, VCR, and NLVR2 add reasoning-oriented settings, while LLaVA-Bench, MME, and MMBench assess generalization and external-knowledge use.Source: §A.2 · p. 25
  • Lens targets spatial reasoning through multi-turn dialogue over human-annotated real-world data, yet advanced MLLMs still face significant challenges on its tasks.Over 80% of its data was collected after September 2024, and the benchmark complements existing evaluations by assessing contextual and cross-modal reasoning.Source: §A.2 · p. 25

A.3. Visual Question Answering with Spatial and Commonsense Reasoning

The section reviews explicit, implicit, and alignment- and chain-of-thought-based approaches to spatial VQA, alongside benchmarks spanning real-world, synthetic, and 3D settings. It presents Lens as a more comprehensive, real-world-oriented assessment of spatial reasoning.

Spatial reasoning approaches

  • Explicit methods use object–relation graphs and geometric priors, while implicit methods learn spatial dependencies through cross-modal attention and multi-scale feature integration.Examples include XNM, NS-VQA, and Neural State Machine [46] for explicit modeling, and LXMERT [113], UNITER [21], and OSCAR for implicit fusion.Source: §A.3 · p. 26
  • Region-level alignment and chain-of-thought prompting support spatial VQA by aligning spatial descriptors with image regions and decomposing complex reasoning into interpretable steps.CLOC [17] uses region–text contrastive learning, while Visual-CoT provides bounding-box and reasoning annotations for supervised training.Source: §A.3 · p. 26

Benchmark datasets

  • Spatial VQA benchmarks span real-world images, controllable synthetic scenes, and 3D scans or point clouds, supporting evaluation of varied spatial reasoning capabilities.Examples include GQA [47], VQA v2 [36], and TDIUC [55] for 2D real-world settings; CLEVR [54] and CLEVR-Humans for synthetic settings; and ScanQA [6] and 3D-VQA for 3D contexts.Source: §A.3 · p. 26

Lens benchmark

  • Lens combines spatial relations, visual reference resolution, and 3D scene comprehension with multi-step reasoning in real-world-aligned scenarios.Unlike traditional benchmarks focused solely on positional relations, it aims to assess complex spatial semantics while mitigating the domain gap between synthetic data and applications such as robotic navigation, AR/VR, and cross-view retrieval.Source: §A.3 · p. 26

A.4. Visual Reasoning in Creative Advertisement Videos … ZLC Team

The subsection presents creative advertisement video reasoning as inference of latent thematic meaning from multimodal evidence, and positions AdsQA and the VR-Ads track as a benchmark for this higher-level reasoning.

A.4. Visual Reasoning in Creative Advertisement Videos

  • Creative ads are short but rich in abstract cues, requiring emotion, metaphor, and cultural-symbol reasoning beyond explicit visual recognition.The reasoning integrates multimodal evidence to infer latent thematic meanings.Source: §A.4 · pp. 26–27
  • Existing video QA benchmarks emphasize actions, temporal sequencing, narrative dependencies, or causal reasoning, but remain limited in evaluating abstract semantics and creative content.Examples include MSRVTT-QA, TGIF-QA [50], TVQA [64], MovieQA, NExT-QA, AGQA [37], and STAR [123].Source: §A.4 · p. 27
  • Prior ad-understanding work targets specific dimensions, while Pit [48] has limited accessibility, diversity, and QA format; no widely adopted comprehensive video-ad QA benchmark exists.Related tasks include persuasion strategy analysis, image ad retrieval, intent understanding [51], and visual metaphor comprehension [4].Source: §A.4 · p. 27
  • AdsQA evaluates five dimensions: visual concepts, emotion, theme and message, persuasion strategy, and target audience recognition.It spans low-level perceptual cues and high-level abstract semantics.Source: §A.4 · p. 27
  • AdsQA frames visual, textual, and sometimes auditory integration as a way to infer communicative intent, and VR-Ads advances reasoning beyond explicit perception tasks.The track raises demands for cross-modal fusion, temporal modeling, external knowledge integration, and interpretability.Source: §A.4 · p. 27
  • The benchmark has implications for advertisement understanding, content moderation, user profiling, and recommendation systems.Source: §A.4 · p. 27

Related papers

Loading 2509.14142v1…