PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometric Inconsistency Localization in Multi-View Image Sets

Xander Staelens, Albéric Loos, Bert Ramlot, Hannes Mareen, Peter Lambert, Glenn Van Wallendael

arXiv:2609.31247v1cs.CVcs.AIcs.CRcs.MM

TL;DR

Multi-view geometric consistency is promising for forensic analysis, but remains underexplored for wide-baseline image pairs, and existing methods and datasets are limited. The paper introduces DeformView and DEFECt3R, showing improved localization with fewer false positives than consistency-scoring methods. The findings also identify a boundary: the dataset uses controlled deformations rather than inconsistencies from AI-generated multi-view imagery.

  • Problem

    Multi-view consistency has received limited attention in multimedia forensics, particularly for wide-baseline pairs, while existing approaches and datasets do not provide suitable pixel-level localization evidence.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces DeformView and DEFECt3R, a supervised pixel-level classifier using aligned cross-view features and hard negatives.

    Source: §1 · p. 2

  • Results

    DEFECt3R substantially reduces false positives and improves localization performance, while feature representation and correspondence quality both contribute to performance.

    Source: §6 · p. 6

  • Takeaways & Limitations

    Multi-view geometric consistency can support future multimedia forensic methods when correspondence estimation and inconsistency modeling are addressed jointly.

    Source: §6 · p. 7

    DeformView uses controlled geometric deformations rather than deformations in AI-generated multi-view imagery, so transfer to those inconsistencies remains to be evaluated.

    Source: §6 · p. 7

Abstract

from arXiv · show

Novel view synthesis (NVS) models can produce realistic new views of the same scene from different viewpoints. However, these generated views are not always geometrically consistent with one another. Multi-view (MV) consistency has shown promise as a tool for evaluating these NVS models. Its potential for multimedia forensics, however, remains largely unexplored, particularly for localizing geometric inconsistencies across wide-baseline image pairs. To enable research in this direction, we introduce DeformView, a wide-baseline MV dataset with pixel-level annotations of geometric inconsistencies. Using DeformView, we evaluate state-of-the-art MV consistency-scoring methods and show that approaches developed for NVS evaluation transfer poorly to the forensic task of geometric inconsistency localization. To address this limitation, we propose DEFECt3R, a lightweight learning-based classifier that uses cross-view feature relationships to localize geometric inconsistencies at the pixel level. By learning from explicit supervision, including hard negatives from geometrically consistent yet deformed views, DEFECt3R improves localization performance and substantially reduces false positives compared to existing consistency-scoring methods. Ablation experiments further show that both feature representations and correspondence quality contribute to localization performance. Overall, our findings demonstrate that MV geometric consistency is a promising yet underexplored signal for multimedia forensics and establish a benchmark and baseline for geometric inconsistency localization in wide-baseline MV image pairs. Code and dataset are available at https://github.com/IDLabMedia/DeformView-DEFECt3R

1 Introduction

As generated imagery and novel views become more realistic, single-image forensic analysis leaves multi-view geometric consistency largely unexplored. The paper introduces a dataset and a pixel-level localization method to address the lack of suitable wide-baseline forensic tools.

1 Introduction

  • Single-image synthetic-image detection does not reason about relationships between views, where geometric contradictions can reveal synthetic origins even when views appear plausible individually.Real photographs share physical geometry, while generative models are not constrained to preserve it.Source: §1 · pp. 1–2
  • Existing multi-view consistency research is limited and centers on NVS or video evaluation, global scores, handcrafted constraints, or highly overlapping sequences rather than wide-baseline forensic pairs.Forensic use requires interpretable pixel-level localization and applicability across substantial viewpoint differences.Source: §1 · p. 2
  • No suitable wide-baseline multi-view dataset provides controlled inconsistencies with pixel-level ground-truth annotations for developing and evaluating localization methods.Source: §1 · p. 2
  • The paper introduces DeformView, DEFECt3R, and a benchmark evaluating consistency-scoring methods for wide-baseline geometric inconsistency localization.DeformView supplies pixel-level annotations; DEFECt3R improves localization and reduces false positives over existing methods.Source: §1 · p. 2

2 Related Work: Multi-View Consistency

Prior multi-view consistency research is constrained by dataset gaps and methods designed mainly for NVS or video evaluation. The paper positions its dataset and localization model as responses to those limitations.

2 Related Work: Multi-View Consistency

  • Multi-view consistency is geometric agreement between views of the same scene, but progress is limited by missing datasets and limitations in existing methods.Source: §2 · p. 2

2.1 Multi-View Datasets

  • Existing datasets do not provide a suitable wide-baseline pixel-localization benchmark: object-level copy-paste labels do not identify geometric-error pixels, while image-space warps lack shared 3D geometry and consistent hard negatives.The latter hard negatives help distinguish cross-view inconsistency from visual deformation.Source: §2.1 · p. 2

2.2 Multi-View Consistency Methods

  • Existing methods are poorly matched to wide-baseline forensic pairs: 3DCS needs many views, TSED requires camera poses and tests epipolar agreement, and MEt3R produces a global score.Source: §2.2 · p. 2
  • GeCo localizes inconsistencies in highly overlapping video frames, but its optical-flow component relies on dense flow and substantial co-visibility that are difficult to satisfy across wide-baseline pairs.Source: §2.2 · p. 3
  • DeformView and DEFECt3R address the related-work gaps with controlled, pixel-labeled wide-baseline data and a model evaluated for geometric inconsistency localization.Source: §2.2 · p. 3

3 Proposed Dataset: DeformView

DeformView creates controlled geometric inconsistencies by deforming 3D meshes, enabling pixel-level ground truth and consistent cross-view hard negatives. Its design supports evaluation of wide-baseline localization methods.

3 Proposed Dataset: DeformView

  • The dataset contains 1029 scanned-object meshes with localized sinusoidal deformations covering 8%–16% of faces and amplitudes of 4%–10% of each object's bounding-box diagonal.Source: §3 · p. 3
  • Objects are rendered from 24 viewpoints, with pairs separated by at most 120° to create a wide-baseline setting while retaining sufficient overlap for comparison.The views use a fixed 20° elevation and 45° field of view; each object yields up to 192 candidate pairs.Source: §3 · p. 3
  • Pixel-level masks come from thresholded differences between corresponding original and deformed renders, and object-level splits prevent objects from crossing training, validation, and test sets.The split is 75% training, 15% validation, and 10% test, evaluating generalization to unseen objects.Source: §3 · p. 3
  • DeformView modifies underlying 3D geometry, enabling precise pixel labels, consistent rendering of the same deformation across viewpoints, and geometrically consistent yet visually deformed hard negatives.These properties distinguish it from cross-view copy-pasting and image-space deformations.Source: §3 · p. 3
  • Positive original-deformed pairs localize inconsistency to the deformed region, while original-original pairs are easy negatives and deformed-deformed pairs are consistent hard negatives.Pairs involving deformations are retained only when the deformed region is visible in both views.Source: §3 · p. 3
  • The controlled inconsistencies, pixel-level labels, and hard negatives make DeformView a benchmark for scoring methods and training dedicated localization models.Source: §3 · p. 3

4 Proposed Method: DEFECt3R

DEFECt3R learns pixel-level geometric inconsistency from aligned cross-view feature differences rather than treating feature dissimilarity alone as proof of inconsistency. It uses explicit supervision, including consistent but deformed hard negatives, to reduce false positives.

4 Proposed Method: DEFECt3R

  • Feature dissimilarity alone cannot establish geometric inconsistency because it can arise from consistent deformations or correspondence errors; DEFECt3R instead learns from aligned feature pairs.Source: §4 · p. 3
  • DEFECt3R aligns upsampled DINO features using MASt3R-estimated pixel correspondences and feeds the DINO and MASt3R feature differences to a lightweight MLP classifier.The DINO backbone and MASt3R descriptors are frozen throughout the experiments.Source: §4 · p. 3, Figure 2 · p. 4
  • The classifier projects 384-dimensional DINO differences to 64 dimensions, concatenates them with 24-dimensional MASt3R differences, and predicts per-pixel inconsistency.The complete classifier has approximately 63k trainable parameters.Source: §4 · p. 3
  • Pixels are classified independently because the aligned features encode geometric and contextual information, avoiding spatial aggregation and keeping the model lightweight.Source: §4 · pp. 3–4
  • Explicit supervision with deformed-deformed hard negatives teaches the classifier to distinguish genuine inconsistency from visual deformation, reprojection artifacts, and consistent deformations.Source: §4 · p. 4

5 Evaluation

The DeformView evaluation tests pixel-level localization and pair-level classification, finding that DEFECt3R outperforms consistency-scoring baselines while feature and correspondence quality affect performance.

5 Evaluation

  • The evaluation measures pixel-level localization and pair-level classification on DeformView; it excludes 3DCS and GeCo because their view-setting requirements differ from wide-baseline pairs.The dataset includes viewpoint differences up to 120°.Source: §5 · p. 4

5.1 Pixel-Level Localization

  • DEFECt3R consistently outperforms MEt3R in pixel-level AUC, F1, and precision while reducing false positives, especially on hard-negative deformed-deformed pairs.The largest false-positive reduction occurs in visually unusual regions that are geometrically consistent.Source: Table 1 · p. 5, §5.1 · p. 5
  • MASt3R features provide a modest localization improvement, while improved correspondences yield larger gains for DEFECt3R; its higher precision comes with lower recall than MEt3R.Ground-truth correspondences reduce MEt3R false positives but leave its overall discriminative performance largely unchanged.Source: §5.1 · p. 5
  • Qualitative examples show DEFECt3R producing cleaner predictions on manipulated regions and suppressing false positives from correspondence errors and consistent deformations.MASt3R pose errors can project non-overlapping regions onto one another, while unusual appearance can mislead MEt3R on consistent deformed-deformed pairs.Source: §5.1 · pp. 5–6

5.2 Pair-Level Classification

  • DEFECt3R substantially outperforms TSED and MEt3R in pair-level classification, while both consistency-scoring baselines achieve AUCs close to 0.5.The baselines’ F1-optimal operating points have recalls near 1.0 but precision close to the fraction of inconsistent test pairs.Source: Table 2 · p. 6, §5.2 · p. 6
  • Pair-level ablations likewise show modest benefit from MASt3R features and larger gains from improved correspondences, but substantial room for improvement remains.Changing MEt3R aggregation has little impact, and ground-truth correspondences only modestly improve TSED, which remains near random performance.Source: Table 2 · p. 6, §5.2 · p. 6

6 Discussion & Conclusion

The discussion attributes existing methods’ weak forensic transfer to their scoring signals and objectives, while DEFECt3R improves localization through supervised learning. The authors also identify a data-scope limitation and call for evaluation on AI-generated multi-view imagery.

Discussion

  • Existing consistency scores target overall view agreement, so localized forensic inconsistencies may have little effect on their global score.Experiments suggest alternative MEt3R aggregation has limited impact, while TSED’s weak performance with ground-truth poses indicates epipolar correspondences alone are insufficient.Source: §6 · p. 6
  • Ground-truth correspondences reduce MEt3R false positives but have little effect on its overall discrimination, while MASt3R correspondence errors remain a false-positive source.The authors therefore note that correspondence quality alone does not explain existing methods’ limitations.Source: §6 · p. 6
  • DEFECt3R learns patterns associated with reprojection errors and visually unusual but consistent regions, reducing false positives and improving localization.Its precision and recall nevertheless remain limited, leaving reliable inconsistency localization an open problem.Source: §6 · p. 6

Discussion & Conclusion

  • The authors argue that multi-view geometric consistency could support future forensic methods if correspondence estimation and inconsistency modeling are addressed jointly.They report that learned localization exploits information beyond simple consistency metrics and partially compensates for correspondence errors.Source: §6 · p. 7
  • DeformView uses controlled geometric deformations rather than AI-generated multi-view imagery, so transfer of the findings to generative-model inconsistencies remains untested.The authors identify adding AI-generated multi-view content and evaluating localization on it as a next step.Source: §6 · p. 7

MASt3R Pose-Estimation GT Poses

Figure 3 presents predictions across consistent and inconsistent DeformView pairs, illustrating false-positive patterns and DEFECt3R’s suppression of many of them.

MEt3R

  • Rows (a)–(c), (d)–(f), and (g)–(i) show consistent original-original, inconsistent original-deformed, and consistent deformed-deformed pairs, respectively.Source: Figure 3 · p. 8
  • Predictions are from view 1, with the object outline shown in yellow.Source: Figure 3 · p. 8
  • The examples include reprojection errors and visually unusual but geometrically consistent regions, where DEFECt3R suppresses many false positives.Source: Figure 3 · p. 8
Loading 2609.31247v1…