PaperlayerView exact revision on arXiv

Source-linked AI summary

Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection

Hatef Otroshi Shahreza, Asif Hussain Khan, Peter Lorenz, Alain Komaty, Sébastien Marcel

arXiv:2608.29802v1cs.CV

TL;DR

PAD and MAD are essential to trustworthy face biometrics, but whether general-purpose FMs and MLLMs contain task-relevant representations and how best to use them is an open question. This study compares five approaches and finds that the pretraining objective associated with best transfer differs by task: self-distillation and discrete-token pretraining favor PAD, whereas large contrastive vision-language pretraining favors MAD.

  • Problem

    Whether general-purpose FMs and MLLMs encode PAD- and MAD-relevant representations, and how to deploy them, remains an open question for defenses essential to trustworthy face biometrics.

    Source: §I · p. 1

  • Method

    The study compares five approaches for using FMs and MLLMs for PAD and MAD, including zero-shot prompting, parameter-efficient fine-tuning, logit probabilities, linear probing, and vision-encoder fine-tuning.

    Source: §I · p. 1, §VII · p. 13

  • Results

    The pretraining objective associated with best transfer differs by task: self-distillation and discrete-token pretraining favor PAD, while large contrastive vision-language pretraining favors MAD.

    Source: §VII · p. 14

  • Takeaways & Limitations

    Strong source-domain separability alone does not ensure robust cross-dataset transfer.

    Source: 2) Linear Probe a Vision-encoder for MAD · p. 10

    Evaluation leaves unseen-attack generalization unresolved: PAD focuses on print and replay attacks, while MAD uses largely fixed morph generators and digital-domain data.

    Source: E. Limitations and Future Work · p. 13

Abstract

from arXiv · show

Face recognition systems are increasingly deployed in security-critical applications, yet they remain vulnerable to presentation and morph attacks. Presentation attack detection (PAD) and morphing attack detection (MAD) are therefore essential components of trustworthy face biometrics. Despite advancements in PAD and MAD methods, existing detectors suffer from limited generalization and degrade in cross-dataset evaluation. In this paper, we systematically investigate whether general-purpose foundation models (FMs) and multimodal large language models (MLLMs) encode PAD-relevant and MAD-relevant information, and how such models can best be deployed for both tasks. We study five approaches with increasing access to the internal information of the model: (i) zero-shot prompting of off-the-shelf MLLMs; (ii) training a shallow model on the next-token logit probabilities at the output of the MLLM; (iii) parameter-efficient fine-tuning on task-specific question-answer data, yielding two specialized MLLMs, called PADLLM and MADLLM, which additionally provide textual reasoning for their decisions; (iv) linear probing of frozen vision encoders; and (v) fine-tuning of vision encoders of FMs and MLLMs. We benchmark 16 open-weight MLLMs and 30 vision encoder backbones on four PAD datasets (MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU) and four MAD datasets (FFHQ, FRGC, FRLL, and FERET). Our experiments show that FMs and MLLMs can achieve significant performance for PAD and MAD. In addition, the fine-tuned models achieve state-of-the-art detection performance in cross-dataset evaluation, indicating that general-purpose pretrained representations carry substantial attack-relevant information. Source code of all our experiments will be publicly released.

I. INTRODUCTION · C MLLM adaptation

The introduction frames PAD and MAD as essential defenses against evolving biometric attacks and cross-dataset failures, then outlines a five-approach study of attack-relevant information in foundation models and MLLMs.

I. INTRODUCTION

  • Presentation attacks exploit image acquisition, while morph attacks combine identities at enrollment; PAD and MAD are essential defenses against both,, –.The paper notes that attacks have evolved and that detectors trained on one dataset often degrade under cross-dataset shifts in sensors, illumination, and attack types –.Source: §I · p. 1

E Vision-encoder adaptation

  • Fine-tuned MLLM vision encoders achieve the study’s strongest cross-dataset detection performance.The study fine-tunes vision encoders from both foundation models and MLLMs for PAD and MAD.Source: C MLLM adaptation · p. 2

D Linear probing

LOGITS LAYER

  • A shallow classifier trained on next-token logit probabilities improves MLLM performance for PAD and MAD.It uses the probabilities instead of decoded text.Source: A Zero-shot benchmark · p. 2

A Zero-shot benchmark

  • The study evaluates five deployment approaches across four PAD datasets and four MAD datasets, benchmarking 16 open-weight MLLMs and 30 vision-encoder backbones.PAD datasets are MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU; MAD datasets are FFHQ, FRGC, FRLL, and FERET.Source: A Zero-shot benchmark · p. 2
  • Zero-shot evaluation prompts off-the-shelf MLLMs with an image and asks for a bonafide-or-attack score.The study evaluates 16 open-weight general-purpose MLLMs with this approach.Source: A Zero-shot benchmark · p. 2

C MLLM adaptation

  • Task-specific question-answer fine-tuning yields PADLLM and MADLLM, which provide attack-detection reasoning and achieve state-of-the-art performance.The adapted models improve performance compared with pretrained baselines.Source: A Zero-shot benchmark · p. 2, C MLLM adaptation · p. 2

II. RELATED WORK · A. Foundation Models

This related-work subsection defines foundation models and distinguishes text-only LLMs from multimodal MLLMs, then surveys their emerging use in face biometrics and security. Prior work spans face recognition, biometric security tasks, general face analysis, and zero-shot evaluation.

II. RELATED WORK · A. Foundation Models

  • Foundation models are pretrained on large-scale data and adapted to a wide range of downstream tasks [39].Source: A. Foundation Models · p. 2
  • Vision backbones span contrastive image–text models such as CLIP, SigLIP, and EVA-CLIP [41], as well as self-distillation, masked-token, reconstruction, and neighbour-based masking approaches [42]–.Source: A. Foundation Models · p. 2
  • LLMs use self-supervised learning on large text corpora, while MLLMs also accept modalities such as images or audio alongside text prompts,,.Source: A. Foundation Models · p. 2
  • Early face-biometric applications included ChatGPT-based face recognition, prediction of soft-biometric attributes, and zero-shot face deepfake detection by Jia et al..A survey reviews FM and MLLM applications in face biometrics.Source: A. Foundation Models · p. 2
  • Shi et al. [52] studied chain-of-thought prompting for face presentation attack and deepfake detection, while Sony et al. [53] benchmarked FMs across broader zero-shot biometric tasks.Source: A. Foundation Models · p. 3
  • Other work benchmarked MLLMs for face recognition, and heterogeneous face recognition [56], adapted CLIP and DINO for face recognition [57], and fine-tuned InternVL3 for general face analysis, [59].Source: A. Foundation Models · p. 3

B. Presentation Attack Detection (PAD) … A. Zero-shot MLLMs

The paper frames PAD and single-image MAD as generalization-challenged tasks, then examines foundation models and MLLMs through five approaches with increasing access to model information. Its zero-shot protocol uses frozen MLLMs to produce bonafide scores from individual face images.

B. Presentation Attack Detection (PAD)

  • PAD methods use texture, color, frequency, depth, and patch-level cues, but compact dataset-specific networks often overfit and degrade under cross-dataset shifts.Cross-dataset evaluation exposes sensor and illumination shifts across MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU; unseen-attack and limited-source protocols assess related robustness conditions.Source: B. Presentation Attack Detection (PAD) · p. 3
  • Prior PAD work adapts foundation models and engineers cross-domain robustness, while this study expands encoder benchmarking to VLMs and MAD datasets against stronger baselines.The cited approaches include FoundPAD, FSFM, FLIP, and DADM; Feng et al. report that DINOv2 with register tokens and PAD-specific augmentations closes much of the gap to larger pipelines.Source: B. Presentation Attack Detection (PAD) · p. 3

C. Morph Attack Detection (MAD)

  • Morph attacks combine two identities into one enrollment image; landmark-based, GAN-based, and diffusion-based generation produce distinct artifacts, with GAN morphs posing comparable or greater matcher threats.MIPGAN uses an identity-prior loss in a StyleGAN generator, while MorDIFF [17] morphs in a diffusion autoencoder’s latent space.Source: C. Morph Attack Detection (MAD) · p. 3
  • Single-image MAD detects morphs from one enrollment image without a trusted live reference, but detectors tuned to one morphing technique generalize poorly to others.OrthoMAD disentangles identity and spoof-relevant features, SelfMAD uses self-supervised pretext tasks, and Zhang et al. probe frozen vision-transformer representations.Source: C. Morph Attack Detection (MAD) · p. 3
  • Prior studies examine zero-shot MLLMs for MAD and language-grounded decisions, but this work addresses the gap in evaluating MLLMs across multiple model-access levels.Ivanovska et al. [83] report emergent morph sensitivity in open MLLMs without MAD-specific training.Source: C. Morph Attack Detection (MAD) · p. 3

III. FOUNDATION MODELS FOR PRESENTATION AND MORPH ATTACK DETECTION

  • The paper presents five approaches to deploy foundation models and MLLMs for PAD and MAD, spanning zero-shot prompting, fine-tuning, logit sampling, and frozen or fine-tuned vision encoders.The approaches are organized by increasing access to the model’s internal information.Source: §III · p. 3, C. Morph Attack Detection (MAD) · p. 3

A. Zero-shot MLLMs

  • Zero-shot detection queries an off-the-shelf, frozen MLLM without task-specific training and converts its answer into a bonafide score based on pretrained priors.Scores range from 0 to 1, with higher values indicating stronger evidence of a bonafide sample; this is the minimum setting for deployment without PAD or MAD training data.Source: A. Zero-shot MLLMs · pp. 3–4
  • Rather than request binary labels, the protocol asks for scores so standard PAD and MAD metrics can be computed, using parallel task-specific prompts and independent image requests.The prompts differ only in attack description; each image is queried independently, and parsed scores in [0,1] prevent information leakage between test samples.Source: A. Zero-shot MLLMs · p. 4

B. Language Model Sampling Logits

The section replaces textual classification scores with next-token logits from MLLMs, using semantically relevant token variants as features for separate PAD and MAD classifiers. It specifies the query prompt and classifier architecture.

B. Language Model Sampling Logits

  • Rather than use a generated classification score, the method extracts raw next-token logits to retain probabilistic information in the MLLM output distribution.The approach is inspired by prior work probing LLM token probabilities for classification [83].Source: B. Language Model Sampling Logits · p. 4
  • For MAD, the prompt requests exactly one label—“yes” for bonafide and “no” for morph attack—with the PAD template substituting “presentation” for “morph.”Given image I and prompt P, the MLLM produces token-level probabilities from its vision encoder and language model.Source: B. Language Model Sampling Logits · p. 4
  • Rather than sampling the full output distribution, the method identifies two disjoint sets of semantically relevant token variants: 15 affirmative variants and 25 negative variants, whose token values form logit vector z.Source: B. Language Model Sampling Logits · p. 4
  • A 40-dimensional vector of affirmative and negative token logits serves as input to separate three-hidden-layer classifiers for PAD and MAD.The classifier has 128, 64, and 32 units, with Batch Normalization, ReLU, and dropout at 0.3 after each hidden layer.Source: B. Language Model Sampling Logits · p. 4

C. LoRA Adaption MLLMs

The section adapts InterVL3 for PAD and MAD using instruction-tuning datasets that combine task-specific QA with general facial descriptions. LoRA updates the language transformer while keeping the vision encoder and pretrained weights frozen, producing separate PADLLM and MADLLM models.

C. LoRA Adaption MLLMs

  • The training data combines task-specific QA pairs with FairFaceGPT’s 87,636 facial-description QA pairs, covering facial attributes, image properties, forensic considerations, and general descriptions.The combined data is intended to support broader facial description and reasoning, not only attack detection.Source: C. LoRA Adaption MLLMs · p. 4
  • Training-split labels generate binary or forced-choice genuine-versus-attack, operational rejection, instrument-or-technique, and broader attack-category questions without language-model assistance.Questions include Live/Spoof or One Identity/Two Identities, and broad categories distinguish paper from display attacks for PAD and landmark-based from other methods for MAD.Source: C. LoRA Adaption MLLMs · pp. 4–5
  • Image-pair QA tests bonafide/attack composition and, for two attacks, shared instruments or techniques and sub-technique location; multiple-choice answers are randomly ordered.The pair questions use four-option choices for composition and sub-technique identification.Source: C. LoRA Adaption MLLMs · p. 5
  • MAD uses 150,502 QA pairs and PAD 55,534; after adding FairFaceGPT, their final instruction-tuning sets contain 238,138 and 143,170 samples, respectively.MAD groups five morphing techniques into three families; PAD groups presentation instruments into printed photo and video replay, each covering two capture devices.Source: C. LoRA Adaption MLLMs · p. 5
  • LoRA fine-tunes InterVL3-8B by freezing its vision encoder and pretrained weights and updating low-rank matrices in language-transformer attention and feed-forward layers, yielding MADLLM and PADLLM.The same LoRA configuration is applied independently to the MAD and PAD datasets.Source: C. LoRA Adaption MLLMs · p. 5

D. Vision-encoder as a Feature Extractor with Linear Probe … C. Models

The paper evaluates frozen and LoRA-adapted vision encoders alongside zero-shot MLLMs for PAD and MAD across diverse datasets and model families. The setup uses ACER with an EER-selected development threshold.

D. Vision-encoder as a Feature Extractor with Linear Probe

  • Linear probing freezes the vision encoder, extracts a fixed-dimensional facial representation, and trains a classifier head for binary attack detection.The approach uses vision encoders from MLLMs and vision foundation models, including CLIP and SigLIP,.Source: D. Vision-encoder as a Feature… · pp. 5–6

E. LoRA Fine-tuning the Vision-encoder as a Feature Extractor

  • LoRA fine-tuning adapts pretrained vision encoders jointly with a classification head while keeping the original backbone parameters frozen.The authors evaluate this parameter-efficient approach across 30 foundation models for PAD and MAD, extending prior CLIP-focused work, including FoundPAD and MADation [38].Source: E. LoRA Fine-tuning the Vision-encoder… · p. 6

A. Dataset

  • PAD evaluation covers MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU, with intra- and cross-dataset protocols across varied attack types and capture conditions.The four datasets span 31,198 test images and include laboratory and mobile scenarios; OULU-NPU has the greatest sensor diversity.Source: A. Dataset · p. 6
  • MAD evaluation uses FFHQ, FRGC, FRLL, and FERET, covering landmark-, GAN-, diffusion-, OpenCV-, FaceMorpher-, and StyleGAN2-generated morphs.FFHQ provides unconstrained images with substantial variation, while FRGC and FERET have controlled acquisition conditions.Source: A. Dataset · p. 6

B. Evaluation metrics

  • ACER is reported in percent, with lower values better; the operating threshold is selected on development data using EER and applied to test data.Source: B. Evaluation metrics · p. 6

C. Models

  • The zero-shot evaluation includes 16 publicly released, instruction-tuned open-weight MLLMs spanning architectural families and capacities from 2B to 13B parameters.Same-family scale pairs and different language-backbone pairs enable comparisons of model capacity and backbone.Source: C. Models · pp. 6–7
  • The study evaluates 30 vision encoders from MLLMs and foundation models, grouped by MLLM, contrastive, self-supervised discriminative, and masked-image pretraining paradigms.The groups include Qwen, LLaVA, and InternVL vision encoders; CLIP, SigLIP, and EVA-CLIP; DINOv2, DINOv3, and MSN; and BEiT and ViT-MAE.Source: C. Models · p. 7

D. Baselines … 1) Zero-shot MLLM for PAD:

The section describes PAD and MAD baselines and evaluates zero-shot MLLMs across PAD and MAD benchmarks. Zero-shot PAD performance is weak on average and varies substantially by dataset.

D. Baselines

  • PAD baselines include DeepPixBiS, FSFM-FAS, FLIP variants, and FoundPAD, spanning pixel-wise DenseNet supervision, face-specific ViT pretraining, and CLIP adaptation.FoundPAD applies LoRA to attention Q,V projections of frozen CLIP ViT-B/16 and ViT-L/14 encoders.Source: D. Baselines · p. 7

1) Zero-shot MLLM for PAD:

  • 31.1% average ACER for LLaVA-NeXT-Mistral-7B and 31.3% for InternVL3-8B were the best zero-shot PAD results on MCIO.Several models were near the 50% chance level, so the paper concludes that off-the-shelf MLLMs prompted for liveness judgments are not reliable PAD systems.Source: Table IV · p. 7, 1) Zero-shot MLLM for PAD · p. 7

D. Baselines

  • MAD baselines include OrthoMAD, IDistill, SelfMAD, and MADation, covering orthogonal identity representations, identity distillation, synthetic morph-generic artifacts, and CLIP adaptation.SelfMAD uses an HRNet-W18 without morph supervision; MADation applies LoRA to attention Q,V projections of CLIP ViT-B/16 and ViT-L/14.Source: D. Baselines · p. 7

V. EVALUATION RESULTS

  • The evaluation separately assesses each method described in Section III and compares against recent PAD and MAD baselines.The listed baselines include four PAD methods and four MAD methods.Source: §V · p. 7, D. Baselines · p. 7

A. Zero-shot MLLMs

1) Zero-shot MLLM for PAD:

  • InternVL3-8B reached 9.2% ACER on CASIA-FASD but 47.3% on Replay-Attack and 39.5% on OULU-NPU; no model was best on all four datasets.The paper says model ordering changes across databases, so selecting a zero-shot detector using one dataset gives no guarantee on another.Source: 1) Zero-shot MLLM for PAD · p. 7

2) Zero-shot MLLM for MAD: … 2) Logit Sampling for MAD:

Zero-shot MLLMs perform poorly on MAD, while logit sampling improves MAD results across all evaluated backbones and improves cross-dataset performance. Its effect on PAD is mixed: it helps two backbones but not the other two.

2) Zero-shot MLLM for MAD:

  • InternVL3-8B had the best MAD zero-shot ACER at 19.4%, followed by Ovis1.5-Llama3-8B at 23.60% and Ovis1.5-Gemma2-9B at 24.67%.Some models, including Qwen2.5-VL-7B and LLaVA-NeXT-Vicuna-7B, were close to random chance; the average ACER was lower for MAD than PAD.Source: 2) Zero-shot MLLM for MAD · pp. 7–8, Table V · p. 8

B. Language Model Logits Sampling

  • The logit-sampling approach trains a separate classifier for each source dataset and backbone, unlike zero-shot evaluation and prior work [83].The comparison evaluated top-performing MLLMs using next-token probabilities to score each image.Source: B. Language Model Logits Sampling · p. 8

1) Logit Sampling for PAD:

  • InternVL3-8B and FaceLLM-8B achieved the lowest PAD Overall Avg ACERs with logit sampling, at 31.0% and 31.3%, respectively.Prior work performed better for Ovis1.5-Llama3-8B and Qwen2-VL-7B-Instruct: 38.1% and 35.9%, versus 41.8% and 37.4% for logit sampling. The approach achieved the lowest intra-dataset ACER for all four backbones, while cross-dataset ACER remained comparable.Source: Table VI · p. 8, 1) Logit Sampling for PAD · p. 8

2) Logit Sampling for MAD:

  • 11.0%, 8.0%, 15.6%, and 16.4% were the lowest Overall Avg ACERs for MAD logit sampling across the four evaluated backbones.FaceLLM showed the largest gain, reducing Overall Avg ACER from 34.4% with Zero-shot to 8.0%.Source: Table VII · p. 8, 2) Logit Sampling for MAD · p. 8
  • MAD logit sampling improved cross-dataset ACER for every backbone, from 21.1–43.0% to 9.5–21.7%, and was competitive with the prior method.It also achieved the lowest intra-dataset ACER for all backbones; the results indicate that gains extend beyond training-source fit.Source: 2) Logit Sampling for MAD · p. 8
  • Logit sampling consistently improved MAD across evaluated backbones, with larger gains than for PAD, particularly in cross-dataset generalization.The reported results indicate that its effectiveness varies by task and backbone.Source: 2) Logit Sampling for MAD · p. 8

C. LoRA Adaptation of MLLMs … 2) Linear Probe a Vision-encoder for MAD:

LoRA-adapted MLLMs improved PAD and MAD results, while frozen vision-encoder probes showed that strong within-dataset performance did not guarantee cross-dataset robustness. The section evaluates both approaches across PAD and MAD datasets.

C. LoRA Adaptation of MLLMs

1) LoRA Adapting MLLM for PAD:

  • Average ACER fell from 31.1% with LLaVA-NeXT-Mistral-7B to 13.6% with PADLLM.PADLLM also improved on every dataset: MSU-MFSD, 29.1% to 16.2%; CASIA-FASD, 9.2% to 8.3%; Replay-Attack, 38.0% to 26.6%; and OULU-NPU, 22.2% to 3.2%.Source: 1) LoRA Adapting MLLM for PAD · p. 9

D. Linear Probe a Vision-encoder as Feature Extractor

2) LoRA Adapting MLLM for MAD:

1) Linear Probe a Vision-encoder for PAD:

  • CLIP ViT-B/16 achieved the lowest cross-dataset PAD ACER at 31.5%, while B/32 reached 31.6%, ahead of ViT-L/14 at 34.0% and ViT-H/14 at 38.7%.B/32 had weaker intra-dataset ACER than B/16 (12.6% versus 5.9%); low intra-dataset error did not reliably predict cross-dataset performance.Source: 1) Linear Probe a Vision-encoder for PAD · p. 9, Table X · p. 10

2) Linear Probe a Vision-encoder for MAD:

  • InternViT-6B's MAD intra-dataset ACER of 0.1% contrasted with 18.5% across six cross-dataset pairs, including 31.1% on FERET when trained on FFHQ.Its errors on FRGC and FRLL were 0.0% and 0.1%, whereas the MADation ViT-L/14 baseline scored 3.5%, 4.2%, and 0.4% on FRGC, FRLL, and FERET; transfer was target-dependent, and low intra-dataset error alone did not ensure robustness.Source: 2) Linear Probe a Vision-encoder for MAD · p. 10, Table XI · p. 10
  • EVA-CLIP-8B achieved 5.0% ACER across eight MAD train/test pairs and 6.4% on cross-dataset evaluations.CLIP ViT-H/14 reached 12.1% and SigLIP-large 15.1%; both outperformed all self-supervised encoders, while InternViT-6B averaged 13.9%.Source: 2) Linear Probe a Vision-encoder for MAD · pp. 9–10, Table XI · p. 10

E. Adaptation of Vision-Encoders as Feature Extractor · 1) Adaptation of Vision-Encoders for PAD:

LoRA-adapted vision encoders achieve strong PAD performance, but cross-dataset transfer—not near-saturated intra-dataset results—distinguishes the strongest models. Performance also varies with the source dataset, and model scale alone does not explain transfer differences.

E. Adaptation of Vision-Encoders as Feature Extractor

1) Adaptation of Vision-Encoders for PAD:

  • 0.3% average intra-dataset ACER is achieved by CLIP ViT-B/32, while many encoders remain below 1%, making cross-dataset transfer the main ranking factor.Intra-dataset PAD performance approaches saturation.Source: 1) Adaptation of Vision-Encoders for PAD · p. 10
  • 0.4% ID ACER versus 19.8% CD ACER for LLaVA-NeXT-7B-ViT illustrates the substantial intra- versus cross-dataset performance gap.The paper identifies robustness to acquisition-domain shift as the principal limitation.Source: 1) Adaptation of Vision-Encoders for PAD · pp. 10–11
  • 3.5% mean ACER when adapted on Replay-Attack versus 21.4% when adapted on Oulu-NPU shows that transfer depends strongly on the source dataset.Both values concern the same LLaVA-NeXT-7B-ViT encoder.Source: 1) Adaptation of Vision-Encoders for PAD · p. 10
  • Comparatively small CLIP encoders outperform several vision towers with multi-billion parameters, so model scale alone does not explain transfer performance.This comparison concerns cross-dataset transfer.Source: 1) Adaptation of Vision-Encoders for PAD · p. 10
  • 19.8% mean cross-dataset ACER is achieved by both LLaVA-NeXT-7B-ViT and CLIP ViT-B/16, followed by DINOv2-registers-giant at 20.4%.Table XII summarizes intra-dataset, cross-dataset, and average performance after LoRA adaptation.Source: 1) Adaptation of Vision-Encoders for PAD · p. 10, Table XII · p. 11

E. Adaptation of Vision-Encoders as Feature Extractor

2) Adaptation of Vision-Encoders for MAD: … B. Generic representations in foundation models versus dedicated detectors

Adaptation of pretrained vision encoders and MLLMs yields strong MAD and PAD results, but transfer and the advantage over dedicated detectors vary by task and training conditions.

2) Adaptation of Vision-Encoders for MAD:

  • CLIP ViT-L/14 leads LoRA-adapted MAD encoders with 3.7% average ACER and 4.9% mean cross-dataset ACER, ahead of CLIP ViT-H/14 at 7.1% CD and EVA-CLIP-8B at 8.4% CD.Most encoders achieve near-zero intra-dataset error, so differences are primarily determined by cross-dataset generalization.Source: Table XIII · p. 11
  • FFHQ-trained models transfer better than FRGC-trained models: InternViT-6B averages 0.5% versus 21.8%, while CLIP ViT-L/14 achieves 1.1% versus 6.3%.Several LoRA-adapted encoders trained on FFHQ transfer to FRGC, FRLL, and FERET with error rates of only a few percent; the asymmetry across encoder families underscores the importance of source-domain coverage.Source: 2) Adaptation of Vision-Encoders for MAD · p. 11

A. How much access to the model is needed?

  • For MAD, a frozen EVA-CLIP-8B linear probe reaches 5.0% ACER and beats the literature baseline, whereas PAD's best linear probe, InternViT-6B at 24.4%, remains 7.1% behind FLIP-MCL.Vision-encoder backbone adaptation improves MAD ACER by 1.3%; for PAD, backbone adaptation is necessary because a frozen encoder is insufficient.Source: A. How much access to the model is… · p. 11, Table XV · p. 12, Table XIV · p. 12
  • Across PAD and MAD, best average ACER reaches 14.9% with LoRA and 16.3% with PADLLM for PAD, versus 2.9% with MADLLM and 3.7% with LoRA for MAD.Zero-shot prompting performs poorly, while parameter-efficient adaptation achieves the best performance on both tasks and outperforms literature baselines.Source: A. How much access to the model is… · p. 11, Table XIV · p. 12, Table XV · p. 12
  • Fine-tuning an MLLM versus a vision encoder differs by 1.4% for PAD and 0.8% for MAD, making the choice depend on computation cost and application rather than accuracy.The reported comparisons are 14.9% versus 16.3% for PAD and 2.9% versus 3.7% for MAD.Source: A. How much access to the model is… · pp. 11–12

B. Generic representations in foundation models versus dedicated detectors

  • Across the full model zoo, LoRA-adapted encoders outperform FLIP-MCL for PAD in 5 of 30 cases, versus SelfMAD for MAD in 16 of 30; frozen encoders reach those baselines in 0 and 1 cases, respectively.Thus, general-purpose representations surpass dedicated designs more broadly for MAD, while PAD success is limited to a handful of adapted backbones.Source: B. Generic representations in… · p. 12
  • PAD baselines FLIP-MCL, FLIP-IT, and FLIP-V reach 22.9, 25.8, and 28.1% cross-dataset ACER, ahead of FoundPAD at 32.6–34.7%.FSFM-FAS and DeepPixBiS have low intra-dataset errors of 4.2% and 3.4% but weak transfer errors of 37.3% and 43.1%, respectively.Source: B. Generic representations in… · p. 12
  • For MAD, MADation ViT-L/14 has 0.2% intra-dataset error but 19.2% cross-dataset ACER, while SelfMAD transfers at 6.5% versus 12.9% source error.The paper attributes SelfMAD's transfer performance to self-supervised morph synthesis rather than its backbone.Source: B. Generic representations in… · p. 12

C. Scale provides separability, not adaptability … VII. CONCLUSION

The paper finds that foundation-model representations support PAD and MAD, but model scale, pretraining objective, adaptation, and deployment needs shape performance. It also identifies limitations in attack coverage, evaluation, and interpretability.

C. Scale provides separability, not adaptability

  • Vision-tower size correlates with frozen overall average ACER for PAD (ρ = −0.36, p = 0.06) and MAD (ρ = −0.34, p = 0.07), but not after LoRA adaptation.Post-adaptation correlations are ρ = −0.11 (p = 0.58) for PAD and ρ = −0.19 (p = 0.32) for MAD.Source: C. Scale provides separability, not… · p. 12
  • Frozen-probe and adaptation rankings diverge: InternViT-6B falls from best frozen PAD encoder (24.4%) to 26th after adaptation (30.0%), while LLaVA-NeXT-7B-ViT rises from 14th to 1st for MAD.Rank agreement is weak (PAD ρ = 0.51, p = 0.004; MAD ρ = 0.33, p = 0.07), so choosing a backbone by linear probing alone selects the wrong model for both tasks.Source: C. Scale provides separability, not… · p. 12

D. Complexity and Computational Cost

  • On InternVL-8B, vision-encoder inference is 264× faster than full-MLLM inference (1.95 ms against 515.3 ms), and encoder adaptation takes 46 min against 9 h.Inference requires 620 against 44,676 GFLOPs; the three MLLM settings have comparable inference costs because they share a backbone.Source: D. Complexity and Computational Cost · pp. 12–13, Figure 2 · p. 13
  • For throughput or edge deployment, the LoRA-adapted vision encoder is the clear choice; fine-tuned PADLLM and MADLLM are justified when human-readable explanations are needed.For PAD, the encoder is more accurate than PADLLM (14.9% against 16.3%) and two orders of magnitude faster; for MAD, it is within 0.8% average ACER of MADLLM at a fraction of the cost. Off-the-shelf MLLM prompting is not deployable given high ACER values.Source: D. Complexity and Computational Cost · p. 13

E. Limitations and Future Work

  • Evaluation leaves unseen attacks, temporal PAD cues, and alternative operating thresholds untested, while MAD results do not establish robustness to unseen morphing methods or print-scan attacks.PAD covers primarily print and replay attacks and evaluates held-out datasets rather than attack instruments; video is processed frame-by-frame, discarding temporal and physiological cues.Source: E. Limitations and Future Work · p. 13
  • Other limitations include a single unchanged LoRA configuration, unevaluated MLLM explanation factuality, unmeasured demographic error differences, and possible pretraining contamination.Publicly distributed datasets include FFHQ, FERET, FRGC, CASIA-FASD, and Replay-Attack, so some images may have appeared in evaluated models' pretraining corpora; zero-shot results warrant this caveat.Source: E. Limitations and Future Work · p. 13

VII. CONCLUSION

  • LoRA-adapted generic encoders reach 14.9% versus 17.3% average ACER for PAD and 3.7% versus 11.3% for MAD, surpassing task-specific architectures overall.Only a small subset of adapted encoders outperform the strongest PAD specialist, while a frozen linear head on EVA-CLIP-8B beats dedicated MAD baselines. PADLLM and MADLLM reduce MLLM average ACER from 31.1% to 13.6% and from 19.4% to 2.9%, respectively; best zero-shot averages are 31.1% for PAD and 19.4% for MAD.Source: §VII · p. 13
  • Pretraining objective matters more than parameter count, and its best transfer is task-dependent: self-distillation and discrete-token prediction favour PAD, while contrastive vision-language pretraining favours MAD.Prototype-based pretraining yields near-chance features for both tasks; DINOv2/v3 and BEiT are among the strongest frozen PAD probes, whereas large contrastive vision-language encoders lead for MAD.Source: C. Scale provides separability, not… · p. 12, §VII · pp. 13–14

APPENDIX A LINEAR PROBE AND LORA FINE-TUNING ON PAD AND … (109) OR TRAINABLE

The appendices specify feature extraction and LoRA adaptation for frozen vision backbones and provide detailed PAD/MAD results across decision-extraction settings, protocols, and train/test pairs.

APPENDIX A LINEAR PROBE AND LORA FINE-TUNING ON PAD AND

  • For linear probing, ViT encoders use final CLS features when available, while several VLM vision towers use mean-pooled patch tokens; LoRA adapts attention query and value projections.CLIP, SigLIP, and BEiT use layer-normalized CLS features, EVA-CLIP uses its final image embedding, and the classification head receives the extracted representation.Source: Appendix A · p. 17, Table XVI · p. 18
  • Appendix A’s full linear-probe and LoRA tables report PAD and MAD performance separately for train–test pairs as well as ID and CD averages.The main paper provides compact summaries, while the appendix tables rank models by overall-average ACER, with the best model at the bottom.Source: Appendix A · p. 17, Table XVII · p. 18, Table XVIII · p. 19

APPENDIX B MLLM LOGIT SAMPLING

  • The logit-sampling appendix compares zero-shot prompting, Ivanovska et al. (I&S) [83], and the paper’s logit-sampling approach across MLLM backbones.Its tables report PAD results for M, C, I, and O protocols and MAD results for FFHQ and FRGC, with ID, CD, and overall averages.Source: Appendix B · p. 17, Table XXI · p. 20, Table XXII · p. 21

APPENDIX C FULL TABLE OF DISCUSSION

  • The logit-sampling tables complement the main paper’s aggregate comparisons with more granular results across datasets and evaluation protocols.They compare zero-shot, Ivanovska et al. (I&S) [83], and logit sampling.Source: Appendix B · p. 17
  • Appendix C provides full PAD and MAD results, including individual train/test outcomes and intra-dataset, cross-dataset, and overall averages.PAD results use train/development and evaluation across the four MCIO datasets; MAD results cover FFHQ, FRGC, FRLL, and FERET.Source: Appendix C · p. 17, Table XXIII · p. 21, Table XXIV · p. 21

(109) OR TRAINABLE

  • The appendix’s linear-probe and LoRA tables cover frozen-backbone PAD/MAD evaluation and vision-encoder adaptation, with test ACER reported for each task.For MAD linear probing, the caption defines ID as the two train=test cells, CD as the remaining six, and the average over all eight cells.Source: Table XVII · p. 18, Table XVIII · p. 19, Table XIX · p. 19

APPENDIX B MLLM LOGIT SAMPLING

  • The PAD logit-sampling table averages results over 16 cells overall, four intra-dataset cells for ID, and 12 cross-dataset cells for CD.For each backbone, the best value among the three decision-extraction settings is bolded, and the authors’ method is additionally highlighted.Source: Table XXI · p. 20
  • The MAD logit-sampling table averages results over eight cells overall, two intra-dataset cells for ID, and six cross-dataset cells for CD.As in the PAD table, each backbone’s best value across the three settings is bolded and the authors’ method is additionally highlighted.Source: Table XXII · p. 21

Related papers

Loading 2608.29802v1…