PaperlayerView exact revision on arXiv

Source-linked AI summary

MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs

Xudong Wang, Hao Wu, Haozhe Hu, Peiran Yin, Xinghao Chen, Yunpu Ma, Wei Zhang, Xiaoyu Shen

arXiv:2610.01434v1cs.CVcs.AIcs.CL

TL;DR

MLLM operation compression often treats attention-layer and shared-FFN computation as unified, although redundancy differs across attention paths and modality-specific channel executions. MWOP prunes these units separately with Taylor-guided selection and recovery. On LLaVA-OneVision-7B, it achieves 1.6× prefill speedup with 99.7% average performance retention, and combines with token compression for further speedup.

  • Problem

    Existing operation compression methods largely treat computation within attention heads and shared FFN channels as unified units, leaving finer-grained redundancy underexplored in MLLMs processing long visual-textual sequences.

    Source: §1 · p. 1

  • Method

    MWOP independently prunes V2V, T2V, and T2T attention paths and modality-specific FFN channel executions using staged Taylor-guided selection and LoRA-based recovery.

    Source: §1 · p. 2

  • Results

    On LLaVA-OneVision-7B, MWOP reduces LLM-side FLOPs by 43.2%, retains 99.7% of dense-model performance across 12 benchmarks, and achieves 1.6× decoder-prefill speedup; with token compression, speedup reaches up to 2.9×.

    Source: §1 · p. 3

  • Takeaways & Limitations

    MWOP preserves near-dense performance while reducing computation, and its composition with token compression further accelerates prefill by reducing work on retained tokens as well as token count.

    Source: §1 · p. 3, §8 · p. 11

Abstract

from arXiv · show

Multimodal large language models (MLLMs) incur substantial inference costs when processing long visual-textual sequences. While existing operation compression methods exploit modality-level redundancy, they largely treat computation within attention heads and shared feed-forward network (FFN) channels as unified units, leaving finer-grained redundancy underexplored. We find that redundancy varies both across modality-interaction paths within the same attention head and across visual and textual executions of the same FFN channel. Based on these findings, we propose Modality-aware Width-wise Operation Pruning (MWOP), which independently prunes visual-to-visual (V2V), text-to-visual (T2V), and text-to-text (T2T) attention paths within each layer, and separately selects FFN channels for visual and textual inputs. A first-order Taylor criterion guides the pruning process, with FFN importance re-evaluated after attention pruning and LoRA-based recovery training. To translate the resulting fine-grained sparsity into practical acceleration, we further develop path-sparse Triton attention kernels and compact visual-side FFN execution. MWOP preserves the token sequence while reducing attention and FFN computation, making it complementary to token compression and enabling simultaneous reduction of sequence length and per-token computation. On LLaVA-OneVision-7B, MWOP alone achieves a $1.6\times$ prefill speedup with 99.7\% average performance retention across 12 benchmarks. Combined with two representative token compression methods, it further increases their prefill speedups from $2.0\times$ and $1.9\times$ to $2.9\times$ and $2.7\times$, respectively. Results on Qwen2.5-VL-7B further demonstrate its applicability across architectures. The code is available at https://github.com/EIT-NLP/MWOP.

1 Introduction

MLLM operation pruning has largely treated computation within attention layers and FFNs as unified, despite redundancy varying across attention paths and modality-specific FFN executions. MWOP independently prunes attention paths and visual/text FFN executions, achieving prefill acceleration while preserving near-dense performance and complementing token compression.

1 Introduction

  • Existing operation compression methods leave finer-grained redundancy underexplored because they largely treat computation within attention layers and FFNs as unified units.Attention paths differ in redundancy and pruning sensitivity, while visual-side FFN executions are more redundant than text-side executions.Source: §1 · p. 2
  • MWOP uses first-order Taylor-guided, staged attention-path pruning, recomputes modality-specific FFN importance after attention pruning, and uses LoRA recovery to assess configurations.Path-sparse Triton attention and compact visual-side FFN execution implement the masks without online selection overhead.Source: §1 · pp. 2–3
  • 43.2% fewer LLM-side FLOPs, 99.7% average performance retention across 12 benchmarks, and 1.6× decoder-prefill speedup; token-compression combinations reach up to 2.9×.The reported pruning preserves all text-side FFN executions; combining MWOP with token compression adds 1.4–1.5× speedup.Source: Figure 1 · p. 2, §1 · p. 3
  • MWOP reduces computation on retained tokens while preserving the token sequence, making it complementary to methods that reduce token count.The framework independently prunes V2V, T2V, and T2T paths and selects FFN channels separately for visual and textual inputs.Source: §1 · pp. 2–3

2 Related Work

Related work reduces visual-token redundancy at different stages and compresses operations by skipping or sparsifying computation. MWOP is distinguished by finer-grained width compression across attention paths and modality-conditioned FFN executions.

2 Related Work

  • Visual token compression occurs during visual encoding, between the encoder and LLM, or within the LLM, using pruning or merging strategies.PruMerge and VisionZip merge similar tokens, while FastV and PyramidDrop use text-to-visual attention-derived importance scores.Source: §2 · p. 3
  • Operation compression preserves the token sequence while skipping or sparsifying attention and FFN operations, with methods targeting depth or width.ShortV and ViCA skip visual-token updates in selected layers or throughout the LLM; RedundancyLens uses local V2V attention and dynamic visual-side FFN selection.Source: §2 · p. 3
  • MWOP performs finer-grained width compression through head-wise path-level attention pruning and modality-conditioned selection of shared FFN channel executions.It remains compatible with token compression.Source: §2 · p. 3

3 MWOP: Formulation and Overview

MWOP treats modality-interaction paths within attention heads and modality-specific executions of shared FFN channels as separate pruning units. Taylor-guided staged masks are followed by modality-specific FFN scoring and LoRA-based recovery, while the token sequence is preserved.

3 MWOP: Formulation and Overview

  • MWOP independently masks each head’s attention paths and selectively executes shared FFN channels for visual tokens, textual tokens, or both, while preserving the token sequence.The design is compatible with token pruning.Source: Figure 2 · p. 4
  • With visual tokens preceding user-text tokens under causal attention, each head has V2V, T2V, and T2T paths, supporting visual aggregation, cross-modal transfer, and textual propagation.This functional progression motivates the staged pruning order V2V → T2V → T2T.Source: §3 · pp. 3–4
  • Path-level masking gates each path’s additive contribution without renormalizing retained attention weights, making each head–path pair a pruning unit.T2V and T2T share softmax normalization.Source: §3 · p. 4
  • After attention masks are fixed, MWOP recomputes Taylor importance and selects FFN channels separately for visual and textual tokens, then applies LoRA-based recovery.Shared parameters remain, but visual-side and text-side executions of a channel are separate pruning units; the pruned FFN uses the corresponding parameter slices.Source: §3 · pp. 4–5

4 Path-Wise Redundancy in Self-Attention

Path-wise redundancy varies by task and path, so independent path masks and importance-guided selection outperform whole-head or random alternatives in the reported comparisons. After recovery, the selected asymmetric path configuration retains near-dense performance across task families.

4 Path-Wise Redundancy in Self-Attention

  • Only 126 of 784 heads on General and 141 on Grounding are shared across the three bottom-50% path-importance sets, so low importance on one path does not imply whole-head redundancy.Grounding degrades sharply beyond 40% V2V masking, while retaining over 90% baseline performance at 70% T2V masking; T2T masking causes collapse near 80% on General and 70% on Grounding.Source: §4 · p. 5, Figure 3 · p. 5
  • At 30% masking, Taylor-guided path-level selection retains 95.3% on Grounding versus 49.3% for whole-head masking, and 99.6% versus 96.1% on General.Taylor selection also outperforms random selection at both granularities at this ratio.Source: §4 · pp. 5–6, Figure 3 · p. 5
  • (40%, 60%, 10%) V2V/T2V/T2T masking achieves 100.0% ATR and 99.5%–100.3% retention across task families after LoRA recovery.Raising T2T masking to 40% lowers Reasoning retention to 97.1%; randomized masks at the selected ratios yield 89.4% ATR.Source: §4 · p. 6

5 Modality-Aware Redundancy in the FFN

Modality-specific FFN pruning is better supported than shared channel masking, and visual and text executions show asymmetric redundancy. After recovery, 50% visual-side masking with dense text-side FFNs retains near-dense performance.

FFN Redundancy Is Modality-Conditioned

  • At matched per-modality ratios, Vision+Text masking retains more General and Grounding performance than Global masking, with a widening advantage at higher ratios.Grounding is less tolerant than General of aggressive visual-side masking.Source: §5 · pp. 6–7, Figure 4 · p. 7
  • Visual-side masking preserves General performance over a broad range, whereas text-side masking degrades earlier on both task families.The comparison is training-free and measures performance relative to MWOPAttn with dense FFNs.Source: §5 · p. 6, Figure 4 · p. 7

Granularity and Importance-Guided Selection Both Matter

  • At 50% masking, Taylor selection gains 50.5%, 16.2%, and 47.1% over random selection for All, Vision, and Text masking, respectively.Visual-only Taylor masking is nearly lossless, compared with 83.1% retention under random selection.Source: §5 · p. 7, Figure 4 · p. 7

Recovering the Joint Attention–FFN Operating Point

  • With dense text-side FFNs, 50% visual masking retains 99.7% ATR and 98.9%–100.4% across task families after LoRA recovery.Increasing visual masking to 60% reduces OCR and Grounding retention; 70% lowers General retention to 94.4%.Source: Table 2 · p. 7, §5 · p. 8

6 Experiments

Experiments on LLaVA-OneVision-7B compare MWOP with operation and token compression methods across 12 benchmarks, with additional evaluation on Qwen2.5-VL-7B. MWOP retains near-dense performance while reducing computation, and combinations with token compression improve performance retention and prefill speedups.

6.1 Experimental Settings

  • Experiments use LLaVA-OneVision-7B and Qwen2.5-VL-7B across 12 benchmarks in four capability categories, with one-epoch LoRA recovery on a 282K-sample multimodal instruction mixture.The categories are General, Reasoning, OCR, and Grounding; all methods evaluated with recovery share the same recovery dataset.Source: §6.1 · p. 8

6.2 Main Result

  • At approximately 60% retained LLM-side FLOPs, MWOP achieves 99.7% ATR and 99.6% OGR with the fewest FLOPs among compared operation-compression methods.It has the highest ATR, GRR, and OGR, at 99.7%, 99.9%, and 99.6%, respectively, while retaining 56.8% of FLOPs.Source: §6.2 · pp. 8–9, Table 3 · p. 9
  • At approximately 30% retained LLM-side FLOPs, MWOPP and MWOPZ outperform all baselines overall while using fewer FLOPs.MWOPP and MWOPZ retain 30.8% and 27.9% of FLOPs, respectively; their ATR values are 97.6% and 96.5%.Source: §6.2 · pp. 9–10, Table 4 · p. 9
  • On LLaVA-OneVision-7B, MWOP reduces FLOPs by 43.2% with a 1.6× prefill speedup; on Qwen2.5-VL-7B, it reduces FLOPs by 32.6% with a 1.3× speedup.Combining MWOP with token compression raises LLaVA-OneVision speedups to 2.7× and 2.9×, and Qwen2.5-VL speedups to 1.7× and 2.0×.Source: §6.2 · pp. 10–11, Table 5 · p. 10

7 Discussion

Task-specialized path masks permit stronger compression on selected paths while retaining target-category performance. MWOP also retains dense-model performance on multi-image and video inputs after recovery on single-image data alone.

7 Discussion

  • Task-specialized masks adjust path-compression ratios from the shared configuration using task-specific Taylor profiles before post-training recovery.The shared starting configuration is (40% V2V, 60% T2V, 10% T2T).Source: §7 · p. 11
  • With task-specialized masks, 70% V2V compression retains 100.1% General performance (vs 100.3% with shared masks); retaining all T2T paths raises Reasoning retention from 99.5% to 100.0%.For OCR and Grounding, 20% and 50% T2T compression yield 100.1% and 100.3% retention, versus 99.9% and 100.3%, respectively.Source: §7 · p. 11
  • 100.4% multi-image and 99.2% video retention are achieved after recovery solely on single-image data, with the highest video retention among compared compression methods.The evaluation uses no additional multi-image or video fine-tuning.Source: §7 · p. 11, Table 6 · p. 11

8 Conclusion

MWOP targets distinct redundancy in attention paths and modality-specific FFN executions, combining Taylor-guided pruning with post-training recovery. The accompanying attention analyses examine path importance, its layer-wise profiles, and the evidence supporting path-wise pruning.

8 Conclusion

  • MWOP combines path-level attention pruning and modality-conditioned FFN channel pruning through Taylor-guided selection and post-training recovery.The conclusion reports experiments on two MLLM architectures and complementary gains when MWOP is composed with token compression.Source: §8 · p. 11

A More Attention Analysis

  • The attention analyses compare importance criteria, layer-wise profiles, path-conditional redundancy, cross-task ranking similarity, and calibration-task effects.Together, these analyses provide empirical support for path-wise pruning and inform importance criteria and calibration data.Source: A More Attention Analysis · p. 17

A.1 Attention Importance Estimation

  • Attention paths are treated as separate pruning units, and the analysis distinguishes attention-weight statistics, output contributions, and loss-based importance.Attention contribution is measured before the output projection; first-order Taylor importance approximates the loss change from gating a unit's output to zero.Source: §A.1 · pp. 17–18

A.2 Layer-Wise Distribution of Path Importance

  • V2V importance concentrates around layers 9–17, T2V around layers 13–17, and T2T is broader, with prominent heads around layers 16–21.The differing profiles motivate sequential selection of V2V, then T2V, then T2T, rather than uniform layer-wise allocation.Source: §A.2 · pp. 18–19, Figure 7 · p. 19

A.3 Path-Conditional Redundancy across Task Categories

The analyses find that attention-path redundancy and importance rankings vary across paths and tasks. They also examine calibration choices and modality-conditioned FFN importance, including how attention pruning changes FFN rankings.

A.3 Path-Conditional Redundancy across Task Categories

  • At 30% masking on Grounding, path-wise selection retains 95.3% performance versus 49.3% for whole-head selection.Only 32.1%–49.5% of each path's low-importance set is shared across all three paths, and Grounding retains over 90% at 70% T2V masking.Source: §A.3 · pp. 19–20, Figure 9 · p. 20

A.4 Cross-Task Similarity of Attention Importance

  • V2V Taylor-importance rankings have cross-task Spearman correlations of 0.88–0.98, compared with 0.70–0.93 for T2V and 0.55–0.91 for T2T.Reasoning's T2T correlations with General, OCR, and Grounding are 0.55, 0.60, and 0.57.Source: §A.4 · pp. 20–21, Figure 11 · p. 21

A.5 Effect of the Calibration Task

  • OCR-guided ranking achieves 99.9% ATR, while All Tasks achieves 100.0% ATR and a 99.5%–100.3% category-wise retention range.Reasoning-guided ranking yields 96.6% ATR and 90.3% OCR retention in this comparison.Source: Table 7 · p. 21, §A.5 · p. 21

B More FFN Analysis

  • The FFN analysis compares channel-importance criteria and examines ranking changes, rescoring after attention pruning, and visual-versus-text pruning budgets.Its stated scope spans General, Reasoning, OCR, and Grounding.Source: B More FFN Analysis · p. 22

B.1 Importance of Modality-Conditioned FFN Channels

  • Importance-guided FFN masking generally outperforms random selection; under aggressive text-side masking, Taylor has a clearer advantage on GQA, MMStar, and TextVQA, but not every RefCOCOg ratio.The criteria perform similarly for visual-side masking.Source: Figure 12 · p. 22, §B.1 · p. 23

B.2 FFN Scoring after Attention Pruning

  • Text-token FFN rankings change more than visual-token rankings after attention masking, and rescoring at 40% visual-side masking raises ATR to 100.0% from 94.1%.The recomputed ranking also gains 7.2 percentage points in GRR and 4.6 in OGR.Source: Figure 13 · p. 23, §B.2 · pp. 23–24, Table 8 · p. 23

B.3 Modality-Conditioned Redundancy across Task Categories

Modality-specific FFN masks improve the performance–compression trade-off on several task categories, while visual-side and text-side masking show different sensitivities. Per-benchmark results further reveal that aggregate retention can conceal concentrated task-specific losses.

B.3 Modality-Conditioned Redundancy across Task Categories

  • Vision+Text masks gain an increasing advantage over shared Global FFN masks as the masking ratio rises on General, OCR, and Grounding, while Reasoning curves remain close.The comparison holds the fraction of channel executions suppressed constant for each modality.Source: Figure 14 · p. 24, §B.3 · p. 24
  • Visual-side FFN masking preserves General and Reasoning over a broad range, whereas text-side masking causes earlier declines; aggressive visual masking sharply degrades OCR and Grounding.At moderate text-side masking, General and Reasoning decline more than OCR and Grounding.Source: Figure 15 · p. 24, §B.3 · pp. 24–25

C Additional Results

  • The additional results cover benchmark-level masking outcomes on LLaVA-OneVision-7B, corresponding Qwen2.5-VL-7B results, and input-length effects on computational costs and speedup.They characterize performance retention and efficiency across tasks, backbones, and input profiles.Source: C Additional Results · p. 25

C.1 Detailed Results on LLaVA-OneVision-7B

  • Raising T2T masking from 10% to 40% lowers MMStar from 60.5 to 58.1, while three grounding scores remain stable or improve.At 40% V2V masking, Taylor exceeds random selection by 6.5 points on DocVQA and 6.9 points on RefCOCO+ under identical masking and recovery settings.Source: §C.1 · p. 25, Table 9 · p. 25
  • Increasing V2V masking from 30% to 60% lowers DocVQA from 87.1 to 83.6, while GQA and VQAv2 remain nearly unchanged.Increasing T2V masking from 50% to 80%, with V2V fixed at 40%, lowers DocVQA from 85.7 to 81.9 and TextVQA from 74.7 to 72.0.Source: Table 9 · p. 25, §C.1 · p. 25
  • Raising visual-side FFN masking from 60% to 70%, with text-side FFNs dense, lowers OK-VQA from 60.1 to 50.0 while GQA and VQAv2 change little.Increasing text-side masking from 0% to 30%, with visual masking fixed at 40%, lowers ScienceQA from 96.2 to 86.7 and OK-VQA from 60.0 to 54.3, while grounding scores change less.Source: §C.1 · pp. 25–26, Table 10 · p. 26

C.2 Detailed Results on Qwen2.5-VL-7B

On Qwen2.5-VL-7B, modality-aware attention and visual-side FFN masking retain benchmark performance, while combining MWOP with token compression improves comparison results. Measured prefill speedups are larger for the longer video input profile.

C.2 Detailed Results on Qwen2.5-VL-7B

  • 99.5% ATR, 99.7% GRR, and 99.2% OGR are retained with 40% visual-side FFN masking and no text-side masking.The attention mask is fixed at (60%, 70%, 0%); adding 20% text-side masking reduces ATR to 97.6%, with a larger decline in GRR than OGR.Source: §C.2 · p. 26, Table 12 · p. 27
  • Qwen2.5-VL-7B uses more V2V and T2V masking and less visual-side FFN masking than LLaVA-OneVision-7B, so the adopted ratios differ across backbones.The results support selecting each backbone’s operating point rather than transferring one numerical mask configuration unchanged.Source: §C.2 · pp. 26–27
  • MWOPZ raises ATR from 94.9% to 96.2% and OGR from 92.4% to 95.1%; MWOPP raises ATR from 88.5% to 95.6% and OGR from 83.0% to 93.8%.These comparisons are against the corresponding token-only methods with post-training recovery.Source: §C.2 · p. 27, Table 13 · p. 27
  • 99.5% ATR is retained by operation compression alone, and 99.8% ATR by the (60%, 70%, 0%) attention mask.Adding 10% T2T masking to that attention configuration gives 99.5% ATR; at 30% T2T masking, GRR falls to 98.1% versus 99.9% with dense T2T.Source: §C.2 · pp. 26–27, Table 11 · p. 26

C.3 Effect of Input Sequence Length

  • Longer visual sequences raise attention’s share of dense FLOPs, while FFNs remain the larger component at both representative input profiles.For LLaVA-OneVision-7B, attention accounts for approximately 16.7% of total FLOPs at the image profile and 35.7% at the video profile.Source: §C.3 · pp. 27, 29
  • 3.78× versus 2.92× prefill speedup is measured for MWOPZ on LLaVA-OneVision-7B at video versus image profiles, and 2.94× versus 1.98× on Qwen2.5-VL-7B.MWOP alone increases from 1.56× to 1.63× on LLaVA-OneVision-7B and from 1.28× to 1.51× on Qwen2.5-VL-7B.Source: Figure 16 · p. 28, §C.3 · p. 29

D.1 Details of LMMs

  • The implementation-details appendix specifies pruning, recovery, FLOPs calculation, prefill evaluation, post-training data, and training configuration across two backbones and token-pruning combinations.The evaluated models connect distinct vision encoders to autoregressive language decoders; their attention and FFN masks remain fixed during post-training and multimodal prefill.Source: D Implementation Details · p. 29, §D.1 · p. 29

D.2 Configuration Details of Our Method

The appendices specify the adopted modality-specific masks, sequential candidate search and recovery, FLOPs and latency accounting, inference kernels, training data, and comparison methods.

D.2 Configuration Details of Our Method

  • LLaVA-OneVision masks 40% of V2V, 60% of T2V, and 10% of T2T paths independently within heads, plus 50% of visual FFN channels while retaining all text channels.Qwen2.5-VL uses attention-path ratios (60%, 70%, 0%) and FFN masking ratios (40%, 0%) for visual and text tokens.Source: §D.2 · pp. 29–30, Figure 17 · p. 30, Table 15 · p. 30

D.3 Algorithm Implementation

  • The search proceeds through V2V, T2V, T2T, and then visual and text FFN pruning, recovering candidate masks independently for one epoch before final selection.The final choice starts from the final-stage anchor and replaces it only with a candidate having both higher full-benchmark ATR and lower theoretical FLOPs.Source: §D.3 · pp. 31–32

D.4 FLOPs Calculation Details

  • The FLOPs accounting uses representative visual-text input profiles, counts two FLOPs per multiply-accumulate, and totals decoder attention, FFN, and language-model-head costs.System tokens are excluded from FLOPs; counts include FFN alignment padding but exclude normalization, RoPE, softmax, elementwise operations, and tensor movement.Source: §D.4 · pp. 32–33

D.5 Efficient Inference Implementation

  • Path-sparse Triton attention visits retained key intervals with a shared online softmax, while compact visual FFN weights execute for visual tokens and full FFNs execute for text.Latency is measured for batch-one BF16 prefill on an NVIDIA A100-SXM4-40GB after compilation, packing, tuning, and graph capture.Source: §D.5 · p. 33

D.6 Post-training data composition

  • Recovery uses a 282,293-sample mixture that does not overlap the evaluation sets; the specific training mixture has not yet been publicly released.The mixture covers general visual understanding, grounding, reasoning, OCR, and language-only instruction following.Source: §D.6 · pp. 33–34, Table 16 · p. 34

D.7 Training Configuration

  • LLaVA-OneVision candidates are independently finetuned for one epoch with only the multimodal projector and language-model LoRA adapters updated.The vision encoder and base language-model weights remain frozen; training uses LoRA rank 128 and eight A100s.Source: §D.7 · p. 34

E Details of Comparison Methods

  • The comparison methods span token pruning, attention or layer reduction, and FFN pruning; examples include FastV, ShortV, RedundancyLens, and YOPO.Other described approaches include diversity-based selection such as DivPrune and progressive token pruning such as PyramidDrop.Source: E Details of Comparison Methods · pp. 34–35

F.1 Single-image benchmarks

The single-image benchmarks assess visual grounding, question answering, reasoning, OCR, and diagram understanding; the multi-image and video sets test cross-image interpretation and temporal understanding.

F.1 Single-image benchmarks

  • GQA tests compositional visual reasoning, while RefCOCO, RefCOCO+, and RefCOCOg assess localization from referring expressions with differing annotation constraints and expression detail.RefCOCO+ excludes location words, while RefCOCOg uses longer, more descriptive expressions.Source: §F.1 · pp. 36–37
  • VQA-v2 targets image-grounded open-ended answers, OK-VQA adds external or commonsense knowledge, and ScienceQA, AI2D, and MMStar assess multimodal reasoning.MMStar samples are curated to require visual evidence and reduce language-only shortcuts and potential data leakage.Source: §F.1 · p. 36
  • TextVQA and DocVQA test reading text in natural images and documents, while OCRBench covers text recognition, document understanding, and information extraction.DocVQA additionally emphasizes layout understanding and information localization.Source: §F.1 · p. 36

F.2 Multi-image benchmarks

  • Q-Bench2, Mantis-Eval, and BLINK test image-pair comparison, multi-image reasoning and co-reference, and visual or spatial perception, respectively.Mantis-Eval questions require joint interpretation across images.Source: §F.2 · p. 37

F.3 Video benchmarks

  • MVBench, Video-MME, and NExT-QA assess video perception and reasoning across frames, including temporal order, event sequences, and action-related causes.Video-MME spans diverse domains and durations, from short clips to hour-long videos.Source: §F.3 · p. 37
Loading 2610.01434v1…