Source-linked AI summary
Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics
Peifeng Liu, Bin Li, Qingsong Zhang, Yangxin Yu, Leqing Chen, Xiaoye Qiu
TL;DR
Text-centric image forensics needs authenticity detection, spatial grounding, and evidence-based explanation because subtle edits can alter critical document information. This paper combines a detector, difficulty-aware localizer, and evidence-conditioned MLLM reasoner, achieving a final score of 0.638 and second place on the hidden challenge test set.
Problem
Text-centric image forensics must detect authenticity, localize tampering, and explain evidence because small edits can modify critical information in documents and receipts.
Method
An evidence-guided detector-localizer-reasoner framework combines global authenticity priors, difficulty-aware localization, and evidence-conditioned report generation.
Results
0.638 on the hidden test set and second place in the challenge are achieved by the proposed system.
Takeaways & Limitations
Combining detector-derived authenticity priors with localization-based spatial evidence produces more balanced performance across detection, localization, and explanation dimensions.
Abstract
from arXiv · showhide
The rapid progress of AIGC has made text-centric image manipulation increasingly accessible, creating new forensic challenges that require not only authenticity detection but also spatial grounding and evidence-based explanation. This paper presents our solution to the GenText-Forensics Challenge at ACM Multimedia 2026. We propose an evidence-guided detector-localizer-reasoner system, where an image-level detector provides a global authenticity prior, a dedicated localizer extracts tampered regions as spatial grounding evidence, and an MLLM-based reasoner generates structured forensic reports grounded in this expert forensic evidence. These modules are connected through a cascaded evidence flow: the detector gates the subsequent localization and prompting process, the localizer converts tamper responses into grounding boxes, and the reasoner is trained to synthesize the detector decision and localized evidence into the final report. As a key part of our method, we introduce iterative difficulty-aware mining to improve localization quality and apply report-mask consistency post-processing to align report grounding with predicted masks. On the official hidden test set, our system achieves a final score of 0.638 and ranks second in the challenge, validating the effectiveness of the proposed evidence-guided system. The code is available at https://github.com/peifengLiu42/ACMMM26-evidence-guided-detector-localizer-reasoner-system.
1 Introduction
Text-centric image forensics must jointly address authenticity, tampering location, and evidence-based explanation because small edits can alter critical document information. The proposed framework assigns these tasks to complementary expert and language modules connected through evidence-guided processing.
- Motivation: Small text edits can modify critical information while preserving layouts, potentially distorting the evidentiary validity of documents and receipts.The forensic task therefore extends beyond image-level authenticity classification to localization and evidence-based reporting.
- Task Setting: The challenge evaluates a unified structured-report task combining authenticity judgment, spatial grounding, evidence-based reasoning, and report quality.It uses high-resolution, text-rich images with dense layouts and semantic dependencies.
- Motivation: Expert forensic models capture low-level tampering traces but lack semantic report generation, whereas MLLMs reason semantically yet remain limited in subtle artifact perception and precise localization.The heterogeneous evidence spans global authenticity cues and local traces such as edge blending, background discontinuity, and compression mismatch.
- Framework: The framework decomposes forensics into a detector, a difficulty-aware localizer, and an evidence-grounded reasoner that aggregates their outputs into one report.The detector learns global authenticity priors with auxiliary localization supervision, while the localizer mines hard samples and error-prone regions for small edits and ambiguous boundaries.
- Framework: The reasoner uses detector predictions and suspicious-region boxes in an evidence-conditioned prompt, constraining reports to expert outputs and reducing ungrounded claims.The system achieved second place in the final evaluation.
2 Related Work
Prior work separates expert artifact-based forensics from MLLM-centric semantic reasoning. These approaches connect low-level manipulation cues and high-level interpretation through different specialized designs.
- Overview: Image manipulation detection and localization methods broadly divide into expert forensic models focused on low-level traces and MLLM-centric models adding multimodal semantic priors.The distinction concerns the balance between artifact sensitivity and interpretable reasoning.
- Expert Models: Expert models analyze artifacts including boundary inconsistency, noise mismatch, compression traces, statistical anomalies, and frequency-domain cues.Document-forensics methods such as DTD and FFDN use DCT, wavelet, and frequency-domain information.
- MLLM-Centric Models: MLLM-centric methods combine multimodal language models with domain-aware modules, mask-aware extractors, detection or segmentation tokens, and trace encoders.These designs aim to connect low-level forgery cues with high-level semantic reasoning and interpretability.
3 Proposed Solution
The proposed system decomposes text-centric image forensics into detector, localizer, and reasoner modules connected by cascaded evidence flow. It strengthens spatial evidence through auxiliary localization supervision and iterative difficulty-aware mining, then constrains report generation with localized evidence and consistency post-processing.
- Evidence-guided pipeline: The detector first predicts an image-level forgery probability, which serves as a global expert prior and gates subsequent spatial evidence extraction.The predicted label is obtained by thresholding the detector output; localization is activated for images classified as forged.
- Evidence-guided pipeline: For forged predictions, the localizer produces a heatmap that is binarized, connected-component filtered, and converted into bounding boxes for reasoning.The boxes use localization and minimum-area thresholds and serve as explicit spatial expert evidence.
- Evidence-guided pipeline: The reasoner uses detector-gated prompts: forged-oriented reasoning receives localized suspicious regions, whereas authentic-oriented prompts omit forced grounding regions.This design triggers forged-oriented reasoning only when detector predictions and localized expert evidence support it.
- Evidence-guided pipeline: Report-mask consistency post-processing refines generated grounding fields with localization-derived boxes and supplements missing localized regions when necessary.The final report is therefore constrained by expert evidence from detection and localization.
- Detector design: DINOv3 combines the CLS token with mean-pooled patch tokens, while an auxiliary convolutional localization head predicts coarse tampering heatmaps from reshaped patch features.Joint image-level and mask supervision encourages the shared backbone to retain tampering-sensitive spatial cues.
- Difficulty-Aware Tamper Localization: Difficulty-aware localization mines hard samples and error-prone regions using localization errors and cross-model disagreement, then fine-tunes with difficulty-weighted sampling and hard-region crops.The closed-loop process refreshes hard samples after each model update so learning focuses progressively on remaining poorly localized regions.
4 Experiments
Experiments evaluate the evidence-guided system through internal validation, component ablations, and the official hidden challenge test. Results show that combining expert evidence, auxiliary localization supervision, difficulty-aware mining, and report-mask consistency improves balanced forensic reporting.
- Experimental Setup: Experiments use an internal validation split with 10% of real and forged training images held out under random seed 42.The official test annotations were withheld, so component evaluation preceded submission to the official evaluation server.
- Experimental Setup: The evaluation measures detection, localization, and explanation using classification, pixel-wise, BERTScore, and cosine semantic similarity metrics.Detection includes Bal-Acc and Wtd-F1; localization includes precision, recall, F1, and IoU.
- Main Results: The full method achieves the best overall balance across image-level detection, pixel-level localization, and explanation quality.Expert models generally provide stronger localization, while MLLMs provide competitive semantic explanation but weaker fine-grained grounding.
- Ablation Studies: Removing evidence guidance weakens either authenticity judgment or spatial grounding, showing complementary roles for detector and localizer outputs.The full model combines detector-derived authenticity priors with localization-based spatial evidence for grounded report generation.
- Ablation Studies: Removing auxiliary localization loss degrades detection and localization-related performance by weakening tampering-sensitive representations.Coarse mask supervision encourages the detector to preserve local artifact cues while learning global authenticity features.
- Ablation Studies: A second difficulty-mining round improves IoU by 9.53 points over the DTD baseline, whereas a third round causes a performance drop.The final system therefore uses the second-round difficulty-mined localizer.
- Ablation Studies: Removing report-mask consistency post-processing noticeably lowers localization metrics while barely affecting explanation scores.The post-processing aligns report grounding boxes with final masks and supplements missing regions without changing semantic content.
- Final Challenge Submission: 0.638 final score ranks the system second on the official hidden GenText-Forensics Challenge evaluation.The result is reported under the official challenge protocol because test annotations were withheld.
5 Conclusion
The paper presents an evidence-guided detector-localizer-reasoner system for the GenText-Forensics Challenge. Combining its principal components yields a 0.638 hidden-test score and second-place ranking.
- Conclusion: The proposed system combines auxiliary localization loss, difficulty-aware mining, and report-mask consistency post-processing.These components are integrated in an evidence-guided detector-localizer-reasoner pipeline.
- Conclusion: 0.638 on the hidden test set earns the system second place in the challenge.The authors also summarize practical insights from building a coordinated evidence-grounded forensic pipeline.