AD-MIR: Bridging the Gap from Perception to Persuasion in Advertising Video Understanding via Structured Reasoning

1. Introduction

• Advertising’s non-linear narratives use cinematic techniques as vehicles for abstract persuasion, making the link between visual facts and strategic intent central to understanding ads.For example, a fade to black may convey suppression intended to trigger pain points.Source: §1 · p. 1
• General video models can lose details across long narratives and hallucinate strategic motivations, while DVD locates events but does not decode the marketing psychology behind them.ReAd-R is also described as lacking explicit structural constraints for persuasion analysis, leaving it prone to shallow associations.Source: §1 · pp. 1–2
• AD-MIR is designed to bridge pixel-level perception and persuasive intent through structured narrative memory and an iterative inquiry loop that aligns hypotheses with visual anchors.Its two-stage architecture constructs structured memory first, then uses a reasoning agent to connect high-level hypotheses to precise visual evidence.Source: §1 · pp. 2–3
• AD-MIR routes factual inquiries to perception tools and strategic inquiries to communication experts grounded in marketing psychology.This division supports analysis of persuasion, rhetoric, and emotional arcs, rather than relying only on literal visual descriptions.Source: §1 · pp. 2–3
• AD-MIR validates high-level interpretations against specific video frames, requiring visual support rather than linguistic priors.Its visual verification is intended to improve the reliability and interpretability of persuasion analysis.Source: §1 · p. 3

2. Method

• AD-MIR couples structured multimodal memory with a ReAct-style agent that iteratively aligns visual evidence with grounded marketing inference.Its workflow has two stages: memory construction isolates narrative elements, followed by iterative intent reasoning.Source: §2 · p. 3

2.1. Problem Definition

• The task is modeled as a partially observable Markov decision process because the agent cannot perceive the full video in one step.The state includes the multimodal database, context-anchored subject registry, and interaction history; actions follow a prompt-based policy with frozen model parameters.Source: §2.1 · p. 3

2.2. Structure-Aware Memory Construction

• Offline preprocessing decouples raw video into an indexable multimodal database to support long-video reasoning and evidence backtracking.The pipeline discretizes video into clips and generates visual descriptions and timestamped audio transcripts.Source: §2.2 · p. 3, §2.2.1 · p. 3

2.2.1. HYBRID SEMANTIC-LEXICAL INDEXING

• Hybrid indexing combines dense semantic similarity with exact keyword matching to retrieve both abstract concepts and specific marketing clues.The lexical weight is controlled by β; semantic retrieval can capture concepts such as “a warm dinner,” while keyword matching can locate “Nike” or “50% OFF.”Source: §2.2.1 · pp. 3–4

2.2.2. CONTEXT-ANCHORED SUBJECT REGISTRY

• A context-anchored subject registry profiles characters and activates those most relevant to the query and global narrative, suppressing background noise.The system combines the query with a global narrative summary to resolve referential ambiguity before ranking character profiles.Source: §2.2.2 · p. 4

2.3. Structured Reasoning Agent

• The reasoning agent uses frozen LMM parameters in a prompt-guided Think-Act-Observe loop, backtracking when an answer fails visual grounding.A weak-evidence constraint is added to the interaction history to guide re-retrieval.Source: §2.3.1 · p. 5
• Its tools divide visual-evidence retrieval from narrative reasoning, using global browsing, temporally fused clip search, frame inspection, and communication experts.Frame inspection varies sampling by query mode, while communication experts use a 64-frame grid with aligned audio and global summaries to ground persuasion analysis.Source: §2.3.2 · pp. 5–6

2.4. Reasoning Reliability Mechanisms

• The agent zooms from narrative-level hypotheses to relevant clips and pixel-level checks, then rejects answers whose key claims lack frame support.Insufficient grounding triggers forced backtracking and a negative constraint that directs alternative retrieval.Source: §2.4 · p. 6

3.1. Experimental Setup

• On AdsQA, which spans 38 categories from factual retrieval to rhetorical and audience analysis, evaluation uses an LLM judge and strict and relaxed accuracy across five task dimensions.The judge scores predictions 0, 0.5, or 1.0; a refinement module limits outputs to fewer than 30 tokens to address length bias.Source: §3.1 · pp. 6–7

3.2. Main Results

• AD-MIR sets state-of-the-art benchmarks on AdsQA under both strict and relaxed accuracy metrics.The supplied passages do not provide the Table 3 values for the overall comparison.Source: §3.2 · p. 7
• With the o1 backbone, AD-MIR reaches 38.1% strict accuracy (+5.0%) and 60.0% relaxed accuracy (+7.6%); its Qwen2.5-VL-7B version improves over its base model by +4.5% strict and +3.1% relaxed.The reported gains over base models occur across both metrics.Source: §3.2 · p. 7
• AD-MIR (Qwen2.5-VL-7B) attains 52.1% relaxed accuracy, comparable to the commercial o1 baseline at 52.4%.The authors describe the structured architecture and domain-specific tools as a resource-efficient alternative for high-level video reasoning.Source: §3.2 · p. 7
• Against ReAd-R, AD-MIR (Qwen2.5-VL-7B) achieves +5.7% strict and +0.6% relaxed accuracy without parameter updates.The authors characterize this as a training-free approach that uses structured tools to guide a frozen backbone.Source: §3.2 · p. 7

3.3. Ablation Study and Sensitivity Analysis

• Accuracy improves as lexical weight β rises before stabilizing, while performance increases with reasoning steps up to Tmax = 8 and then saturates or slightly declines.The authors connect lexical matching to fine-grained OCR clues and the reasoning budget to multi-hop evidence collection.Source: §3.3 · p. 8
• Removing Hybrid Indexing reduces performance, and removing the Subject Registry particularly harms social-interaction questions through greater interference from irrelevant background characters.The component analysis removes modules one at a time.Source: §3.3 · p. 8
• Disabling the Communication Expert causes the sharpest decline on persuasion-centric dimensions, while removing the Visual Anchor hurts strict accuracy.The authors associate the former with the role of marketing priors and the latter with suppressing plausible but hallucinated reasoning.Source: §3.3 · p. 8

• AD-MIR combines structure-aware memory construction and iterative structured reasoning to decode marketing strategies in non-linear narratives.The authors frame the approach as bridging pixel-level perception and high-level persuasive intent.Source: §4 · p. 8
• On AdsQA, AD-MIR achieves state-of-the-art performance, outperforming end-to-end LMMs and general video agents.Source: §4 · p. 8

Impact Statement

• AD-MIR could assist content moderation and regulatory compliance by decoding implicit marketing intents and detecting misleading or non-compliant persuasive tactics.The impact statement connects this capability to greater transparency in digital media.Source: Impact Statement · p. 9

A. Related Works

• The paper builds on a semiotic view of advertising as engineered persuasion, linking observable cues to symbolic references and persuasive strategies.Its focus is modern long-form, information-dense advertising videos, with marketing and rhetoric knowledge alongside stricter evidence grounding.Source: A. Related Works · p. 11
• Prior long-form video work identifies limited context, temporal fragmentation, and imperfect multimodal integration as challenges, while benchmarks show degradation on multi-hour videos.The passage motivates structured memory, temporal indexing, and iterative evidence collection for advertising intent reasoning.Source: A. Related Works · p. 11
• Tool-augmented video agents support iterative evidence gathering, while hallucination analyses report confident but weakly supported LVLM explanations.The paper situates its approach as an advertising-specific agent with evidence-first reasoning and self-correction.Source: A. Related Works · p. 11

• The section covers baseline testing protocols and the proposed AD-MIR agent’s hyperparameter configurations.Source: B. Implementation Details · p. 11

• Baseline MLLMs are evaluated in AdsQA’s standardized zero-shot setting for fair comparison.Source: §B.1 · p. 11
• End-to-end MLLM baselines receive 16 frames sampled uniformly across each video.For models supporting high-resolution or dynamic-resolution inputs, frames preserve aspect ratio and have a longest side no greater than 1024 pixels.Source: §B.1 · p. 11
• The protocol uses AdsQA’s standard Video QA prompts, with greedy decoding for open-ended generation.Prompts combine a task-defining system instruction, interleaved frames, and the question.Source: §B.1 · p. 11

Video Database Construction

• The video database combines 1-FPS frame extraction, five-second GPT-4o clip captions, a cross-clip subject registry, and BAAI/bge-m3 embeddings.The registry tracks character identities and key objects; embeddings have dimension 1024.Source: §B.2 · p. 12

Agent Hyperparameters

• The agent uses GPT-4o, while the reasoning-intensive COMMUNICATION EXPERT uses o1.Source: §B.2 · p. 12
• The ReAct loop is capped at eight steps, with most queries resolved in four to six.The initial global browse retrieves 40 semantic captions to construct narrative context.Source: §B.2 · p. 12

Clip Search and Expert Analysis

• Hybrid vector-and-keyword search retrieves five to eight relevant clips for fine-grained inspection.The COMMUNICATION EXPERT analyzes up to 64 frames stitched into 2 × 2 grid images.Source: §B.2 · p. 12

Answer Refinement and Retry Mechanism

• GPT-4o-mini refines the final answer to under 30 words while preserving critical entities and attributes.Critical API failures trigger exponential backoff, with a maximum of eight retries.Source: §B.2 · p. 12

Temporal Stagnation Detection & Redirection

• Temporal stagnation triggers forced redirection to the first or last 15 seconds when overlap exceeds 0.6 across more than two distinct prior queries.The controller shifts the agent’s focus to video boundaries after repeated searches of overlapping time ranges.Source: §B.3 · p. 12

Anti-Verification Protocol

• The Golden Source rule requires accepting the Communication Expert’s specific names, metaphors, or strategies rather than rechecking them with frame inspection.The stated rationale is that sparse frame sampling can yield false negatives when verifying high-level semantic findings.Source: §B.3 · pp. 12–13

Visual Anchor Repair

• For questions requiring visual grounding, a repair loop adds concrete observable nouns when the draft lacks anchors found in retrieval evidence.The example changes “a vehicle” to “a red Ferrari.”Source: §B.3 · p. 13

High-Density Sampling

• The expert samples 64 frames uniformly across the video or queried segment to capture fleeting commercial cues.The passage contrasts this with standard approaches sampling 8–16 frames.Source: §B.4 · p. 13

2x2 Grid Stitching

• Consecutive frames are stitched into 2 × 2 grids because directly sending 64 images exceeds most VLM APIs’ file-count limits.The frames form 16 batches of four; each batch becomes one high-resolution grid encoded at high detail to preserve sub-frame OCR readability.Source: §B.4 · p. 13

Temporal Serialization

• The VLM receives a sequence of 16 grid images, and the system prompt specifies their reading order for temporal reasoning.Source: §B.4 · p. 13

Spatio-Temporal Grid Projection

• The projection processes 64 frames while using the request overhead of 16 images, quadrupling the temporal context window.Source: §B.4 · p. 13

C. Core Prompts of AD-MIR

• The prompt suite guides the agent through perception, reasoning, and answer refinement.It is organized around distinct stages of the AD-MIR framework.Source: C. Core Prompts of AD-MIR · p. 13

C.1. Perception & Context Construction Prompts

• Offline perception prompts transform raw video into structured semantic indices through narrative reconstruction and dynamic inspection.Source: §C.1 · p. 13

C.2. Structured Agent Reasoning Prompts

• Online inference prompts manage interactions between the ReAct controller and domain-specific experts.Source: §C.2 · p. 15

ReAct Controller: System Instructions

• The ReAct controller prompt treats the communication expert as its primary source and restricts frame inspection to fallback use.It instructs the agent to finish immediately when the expert answers fully.Source: ReAct Controller: System Instructions · p. 15
• The controller instructions require a brief plain-text thought before the agent calls an appropriate tool.Source: ReAct Controller: System Instructions · p. 15

C.3. Data Preprocessing Prompts

• Preprocessing prompts structure video through narrative reconstruction, entity tracking, and fine-grained text extraction.Source: §C.3 · p. 16

C.4. Evaluation Prompts

• The evaluation prompt uses an LLM-as-a-Judge mechanism to compare performance with ground-truth annotations.Source: §C.4 · p. 18

• The instructions prohibit inventing objects that are not present in the frame grids.Source: Communication Expert: Visual Semiotics… · p. 15
• The analysis is tasked with uncovering video narrative structure, character relationships, and persuasive strategy.Source: Communication Expert: Visual Semiotics… · p. 15
• Each 2x2 grid contains four chronological frames, read top-left, top-right, bottom-left, then bottom-right.Source: Communication Expert: Visual Semiotics… · p. 15
• The protocol prioritizes on-screen text as factual and calls for identifying logos.Source: Communication Expert: Visual Semiotics… · p. 15
• Character interactions are analyzed as transactional, conflictual, or affectionate.Source: Communication Expert: Visual Semiotics… · p. 15
• The narrative arc is analyzed as hook, problem, product, and call to action.Source: Communication Expert: Visual Semiotics… · p. 15