PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Single Videos: Benchmarking and Active Evidence Seeking for E-Commerce Cross-Video Reasoning

Jinghan Zhao, Yiman Hu, Liang Wu, Jian Xu, Bo Zheng

arXiv:2610.03099v1cs.CVcs.AI

TL;DR

E-commerce video benchmarks largely assess single videos, although consumers and merchants compare dense multimodal information across advertisements. The paper introduces AdsCVR and AdSeek, using active evidence acquisition and trajectory rectification for cross-video reasoning. AdSeek reaches 74.30% accuracy on AdsCVR, 27.90 percentage points above its Qwen3-VL-8B-Instruct baseline, and transfers to CrossVid.

  • Problem

    Existing advertising benchmarks predominantly focus on single-video understanding, while consumers and merchants need to compare dense multimodal information across videos.

    Source: §1 · p. 1

  • Method

    AdSeek actively selects and aligns cross-video evidence, while offline teacher-guided trajectory rectification supplies missing evidence and corrects faulty reasoning for SFT.

    Source: §1 · p. 2

  • Results

    74.30% accuracy on the AdsCVR test split, 27.90 percentage points above Qwen3-VL-8B-Instruct, with transferability to open-domain CrossVid.

    Source: §1 · p. 3

  • Takeaways & Limitations

    AdsCVR provides an evaluation benchmark for fine-grained commercial reasoning and comparisons across information-dense e-commerce videos.

    Source: §1 · pp. 1–2

    Multi-turn tool calls and repeated context processing can increase inference latency, while RL rollouts and teacher-guided rectification add training overhead.

    Source: §6 · p. 11

Abstract

from arXiv · show

E-commerce videos are information-dense and frequently compared by consumers evaluating products and merchants assessing marketing strategies. However, existing multimodal models mainly focus on single-video understanding and have limited ability to compare information across videos. We introduce AdsCVR, the first e-commerce cross-video reasoning benchmark, containing 2,483 videos and 6,110 question-answer pairs across six reasoning dimensions. Cross- video reasoning requires models to locate fine-grained evidence among many redundant frames and integrate visual details, speech, and on-screen text. We therefore propose AdSeek, an agentic framework that dynamically selects visual and audio tools during multi-turn exploration, replacing static uniform sampling with active evidence acquisition. To address the sparse credit assignment of reinforcement learning, we develop an offline trajectory rectification mechanism that identifies reasoning errors and missing multimodal evidence in RL-generated trajectories. The corrected trajectories provide supervised fine-tuning signals that reduce biases learned during RL. This mechanism supports a rectified bootstrapping pipeline in which initial RL exposes reasoning bottlenecks, supervised fine-tuning corrects them, and a final RL stage further improves the policy. AdSeek achieves 74.30 percent accuracy on the AdsCVR test split, outperforming its Qwen3-VL-8B-Instruct backbone by 27.90 percentage points. It also generalizes to the open- domain CrossVid benchmark, demonstrating effective active evidence gathering.

1 Introduction

The paper introduces AdsCVR to evaluate cross-video e-commerce reasoning and AdSeek to gather evidence actively across videos. It also addresses sparse RL feedback with offline trajectory rectification and reports improved AdsCVR performance and transfer to CrossVid.

1 Introduction

  • AdSeek dynamically selects videos, temporal segments, modalities, and visual regions, then filters irrelevant content and aligns evidence for each comparison objective.The framework uses multi-turn reasoning and tool use for active evidence acquisition.Source: §1 · p. 2
  • Sparse outcome rewards provide little feedback on intermediate RL decisions, so the paper rectifies trajectories with missing evidence and faulty reasoning before further policy refinement.A teacher identifies intermediate deviations; corrected trajectories supervise evidence acquisition and reasoning through SFT, within a pipeline of initial RL, rectification and SFT, then further RL.Source: §1 · pp. 2–3
  • AdsCVR is an e-commerce cross-video benchmark with 2,483 videos and 6,110 QA pairs for comparing fine-grained differences in information-dense advertisements.Decisive evidence can be distributed across videos, non-contiguous intervals, and visual and audio modalities.Source: §1 · pp. 1–2
  • 74.30% accuracy on AdsCVR, 27.90 percentage points above Qwen3-VL-8B-Instruct, with transferability to open-domain CrossVid.The reported AdsCVR result is on the test split.Source: §1 · p. 3

2 Related Work

Prior video benchmarks expanded from single-video tasks to collections requiring distributed-evidence aggregation, while advertising QA benchmarks remained focused on individual videos. AdsCVR connects advertisement understanding with cross-video comparison, and related work describes a shift from uniform sampling toward adaptive evidence acquisition.

Video Understanding Benchmarks

  • Video-collection benchmarks such as CrossVid, MVU-Eval, and CVBench evaluate aggregation of distributed evidence and comparison across videos.The accompanying benchmark table defines temporal and intent coverage and task formats including MCQ, open-ended QA, segmentation, and classification.Source: Video Understanding Benchmarks · p. 3, Table 1 · p. 3

E-commerce Advertisement Video Understanding

  • Earlier and recent advertising benchmarks evaluate videos individually, whereas AdsCVR requires aligning cross-ad evidence to compare product presentations.Its comparison table tracks temporal and intent coverage alongside task format.Source: E-commerce Advertisement Video… · p. 3, Table 1 · p. 3

MLLMs for Video Understanding

  • Uniform frame sampling can omit brief decisive events; agentic approaches instead retrieve or select keyframes adaptively using query relevance and reasoning feedback.AgentCVR extends active acquisition to cross-video reasoning through script-simulated RL.Source: MLLMs for Video Understanding · p. 4

3 The AdsCVR Benchmark

AdsCVR evaluates fine-grained e-commerce video comparison by grouping comparable advertisements and generating evidence-grounded QA pairs. Its construction combines filtering, multidimensional annotation, dimension-specific grouping, and staged question verification and difficulty filtering.

3.1 Video Filtering and Multi-View Annotation

  • Filtering removes videos with extreme durations or low visual variation and balances categories; multidimensional captions cover events, objects, creative structure, and commercial logic.The filtering process yields 2,483 videos across 86 product categories.Source: §3.1 · pp. 4–5, Figure 2 · p. 4

3.2 Video Grouping

  • Dimension-specific rules group two to four videos to compare visual execution and commercial intent across six reasoning dimensions.The benchmark defines comparable groups using shared contextual dimensions.Source: Figure 2 · p. 4, §3 · p. 4, §3.2 · p. 5

3.3 QA Generation and Verification

  • QA generation uses grouped-video metadata and rationales, then filters questions for text-only answer leakage, evidence verifiability, and difficulty.The final set contains 6,110 QA pairs, randomly split into 3,110 training and 3,000 test examples.Source: Figure 2 · p. 4, §3.3 · p. 5

3 The AdsCVR Benchmark

  • AdsCVR contains 6,110 QA pairs from 2,483 videos across 86 product categories, split into 3,110 training and 3,000 test examples.Comparable video groups share contextual dimensions to expose subtle differences in advertising strategies.Source: §3 · p. 4, §3.3 · p. 5

4 AdSeek

AdSeek frames cross-video answering as iterative evidence acquisition and trains its policy through the Rectified Bootstrapping Pipeline (RBP).

4.1 Task Definition

  • Given a query and video collection, AdSeek iteratively acquires cross-video evidence through perception tools until answering or reaching the interaction limit.Source: §4.1 · pp. 5–6

4.2.1 Policy Optimization

  • Policy optimization uses Group Relative Policy Optimization (GRPO), with rewards combining final-answer correctness and action-format compliance.Source: §4.2.1 · pp. 6–7

4.2.2 Rollout Filtering and Resampling

  • Rollout filtering discards trajectories without effective tool calls and groups with zero reward variance, then resamples up to three times to replenish batches.Source: §4.2.2 · p. 7

4.3 Exploitation: Offline Trajectory Rectification

  • Offline rectification diagnoses evidence gaps and reasoning errors, then adds missing observations or teacher comments to regenerate faulty intermediate reasoning.Source: §4.3 · p. 8

4.3.1 Supervised Policy Optimization

  • SFT uses initially successful and rectified trajectories, applying loss only to assistant tokens while holding trajectories fixed and using no perception tools.Source: §4.3.1 · p. 8

4.4 Training Paradigm: Rectified Bootstrapping Pipeline

  • RBP combines initial RL, teacher-guided offline rectification with SFT, and refined RL to correct missing evidence and faulty reasoning before online optimization resumes.Source: §4.4 · p. 8

5 Experiments

Experiments evaluate AdSeek on AdsCVR and CrossVid, then examine how training data and available actions affect accuracy. The results show strong in-domain performance, transfer to general-domain tasks, and variation across training and action-space conditions.

Training and Evaluation Settings

  • Evaluation uses the AdsCVR test split and CrossVid for out-of-domain generalization, with general MLLMs and AgentCVR among the baselines.The AdsCVR split contains 3,000 test and 3,110 training QA pairs.Source: Training and Evaluation Settings · p. 9

5.1 In-Domain Results on AdsCVR

  • 74.30% accuracy on AdsCVR with public out-of-domain training data; AdsCVR-only training reaches 73.57%, 27.17 points above the Qwen3-VL-8B backbone.The AdsCVR-only model also exceeds the AgentCVR reproduction by 2.52 points and approaches Qwen3.8-27B at 75.15%.Source: §5.1 · p. 9, Table 3 · p. 9

5.2 Out-of-Domain Results on CrossVid

  • 31.55% average accuracy on CrossVid exceeds the Qwen3-VL-8B backbone by 2.08 points and the AgentCVR reproduction by 3.31 points.AdSeek improves over the backbone on six of ten tasks.Source: Table 4 · p. 10, §5.2 · p. 10

5.3 Ablations on Training Data and Action Space

  • AdsCVR-only training outperforms MVU-Eval-only training by 9.50 points (73.57% vs. 64.07%), while mixed-source training reaches 74.30%.Mixed-source gains are concentrated in IC (+2.8 points) and TS (+2.0 points), while CS decreases by 1.0 point.Source: §5.3 · p. 10, Table 5 · p. 10
  • Restricting actions to frame_select or audio_select lowers average accuracy to 63.97% or 65.03%, respectively, below the full model across all six dimensions.Visual-only access performs better on IC (63.8% vs. 58.2%); speech-only access performs better on the other five dimensions.Source: §5.3 · p. 10, Table 5 · p. 10

6 Limitations

AdSeek's multi-turn agentic reasoning has computational trade-offs.

  • Sequential tool calls and repeated context processing can increase inference latency, while RL rollouts and teacher-guided trajectory rectification add training overhead.The paper suggests more efficient tool scheduling and lightweight perception models as future directions, particularly for latency-sensitive applications.Source: §6 · p. 11

7 Conclusion

The paper concludes with AdsCVR, a benchmark for fine-grained e-commerce cross-video comparison, and AdSeek, an active evidence acquisition framework optimized through RBP.

  • AdSeek achieves 74.30% accuracy on AdsCVR and demonstrates transferability to open-domain multimodal reasoning tasks.RBP integrates RL exploration with offline trajectory rectification via SFT.Source: §7 · p. 11

AI Use Statement

The research workflow used generative AI for video annotations, candidate questions, filtering, verification, diagnosis, and evaluation, with author review and stated checks. The appendix describes AdsCVR annotation procedures, selected examples across six dimensions, and a multi-role prompt for video analysis.

AI Use Statement

  • Gemini generated multi-view annotations and candidate AdsCVR questions, while other models supported filtering, verification, trajectory diagnosis, and evaluation.The authors report deterministic filtering, representative-example checks, and review of the final manuscript, experimental claims, and released code.Source: AI Use Statement · p. 11

A Annotation Guidelines

  • AdsCVR annotations compare two to four e-commerce videos across six dimensions and use category-balanced filtering, timestamped ASR/OCR, structured descriptions, and hierarchical question verification.The verifier checks shortcut and difficulty issues, answer completeness, multi-video dependence, logical validity, and multimodal evidence requirements.Source: A Annotation Guidelines · p. 16

A.1 Qualitative Examples Across Dimensions

  • Figures 5–10 present selected examples for Temporal Structure, Creative Strategy, Outlier Detection, Intra-class Comparison, Information Density, and Visual Composition.The examples illustrate event organization, attention and product value, category outliers, related-product comparisons, spoken and visual information density, and shot composition.Source: §A.1 · p. 16, Figure 5 · p. 16, Figure 6 · p. 16

A.2 Multi-view Annotation Prompt

  • The multi-view prompt asks for event, scene, creative, and commercial analyses, including shot actions, spatial details, cinematography, audience, and ad logic.It prioritizes shot-time segmentation, or action, transition, and scene changes when that segmentation is unavailable.Source: §A.2 · pp. 19–20

A.3 Dataset Split and Video-Overlap Audit

The section audits a separately extracted 1k-sample test subset for video overlap with training and compares its accuracy with mixed-source AdSeek across six dimensions. The retained high accuracy supports robustness to cross-video transfer.

A.3 Dataset Split and Video-Overlap Audit

  • The audited 1k-sample subset retains high accuracy, supporting AdSeek’s robustness to cross-video transfer.Table 6 aligns audited-subset and mixed-source AdSeek accuracy across six AdsCVR dimensions; no individual scores are supplied here.Source: §A.3 · p. 20, Table 6 · p. 20
  • The test subset was separately extracted and checked for video overlap with the training partition before evaluation.AdsCVR is randomly split into training and test partitions.Source: §A.3 · p. 20, Table 6 · p. 20

B Implementation Details

The implementation details report the SFT and RL hyperparameter configurations for AdsCVR, including their default parallel processing unit counts.

B Implementation Details

C Video Complexity Metrics

The section specifies temporal and spatial complexity proxies used for video filtering, based on sampled-frame visual variation and JPEG compressibility.

C Video Complexity Metrics

  • Temporal filtering samples at approximately 1 fps and uniformly retains at most 16 frames per video.For T selected frames, the temporal features are normalized DINOv3 representations; both temporal metrics are zero when T ≤ 1.Source: C Video Complexity Metrics · pp. 21–22
  • Higher Dcpx and Vcpx indicate greater cumulative and average visual variation, respectively.The temporal metrics use a neighborhood distance d = 8 between selected frames.Source: C Video Complexity Metrics · p. 22
  • Spatial complexity is estimated as average JPEG-encoded bytes per pixel across the selected frames.The specified encoding uses quality 95, 4:2:0 chroma subsampling, and optimized coding.Source: C Video Complexity Metrics · p. 22
  • Higher Jcpx indicates lower spatial compressibility under the fixed JPEG encoding settings.The metric uses each frame’s encoded byte size and resolution.Source: C Video Complexity Metrics · p. 22

Related papers

Loading 2610.03099v1…