PaperlayerView exact revision on arXiv

Source-linked AI summary

inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado

arXiv:2610.18051v1cs.LGcs.AIcs.CL

TL;DR

The paper addresses how to audit whether samples appeared in LoRA fine-tuning data amid concerns about privacy and transparency. It introduces inLoRA-MINT, which classifies membership using internal and output-related representations. Experiments report consistent membership signals across models and datasets, with the strongest performance from combined heterogeneous features.

  • Problem

    The paper addresses how to determine whether specific samples were used to train a model, a question linked to data privacy, transparency, and accountability concerns.

    Source: §I · p. 1

  • Method

    inLoRA-MINT combines activation statistics, gradient-based sensitivity measures, and output-level signals to train a binary membership classifier for LoRA-fine-tuned LLMs.

    Source: §I · p. 2, §VI · p. 6

  • Results

    Across three benchmark datasets and four models, later layers provide stronger membership signals, and the best performance comes from combining last-three-layer activation statistics with gradient and output-based features.

    Source: §VI · p. 6

  • Takeaways & Limitations

    The authors state that the methodology can also be applied to other LLM adaptation techniques and, more broadly, to other AI models.

    Source: §VI · p. 6

    The discussion and experiments are centered on LoRA-adjusted LLMs.

    Source: §VI · p. 6

Abstract

from arXiv · show

This work investigates the application of the Membership Inference Test (MINT) to large language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about data privacy, leakage, and transparency. Building on our previous LoRA-MINT framework, which relies exclusively on perplexity under a strict black-box setting, we propose an approach based on intermediate model representations, including activations and parameter-dependent signals. These representations enable a MINT classifier to distinguish training from non-training samples without requiring full access to model parameters. We evaluated the method on four LLMs (Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B) fine-tuned on three datasets covering medical, mathematical, and academic reasoning tasks. Results show consistent performance across architectures and domains, achieving scores between 95% and 98%, with robustness to variations in model configurations and data distributions. The proposed approach enables efficient auditing of sensitive training data and supports data provenance and compliance with emerging privacy and AI governance regulations. Although our experiments focus on LoRA-based fine-tuning, the methodology can be extended to other LLM adaptation techniques and, more broadly, to other AI models.

I. INTRODUCTION

The introduction presents inLoRA-MINT as a representation-based approach to auditing whether samples were used to fine-tune LLMs with LoRA. It motivates the work through privacy, provenance, transparency, and accountability concerns, and reports effective distinction between training and non-training samples across models and domains.

I. INTRODUCTION

  • The introduction links sensitive-data protection to user trust and compliance with increasingly stringent social concerns about AI use.It describes transparency and accountability as central objectives of the EU regulatory framework and related auditing guidance.Source: §I · p. 1
  • inLoRA-MINT extends perplexity-based LoRA-MINT with internal activations, gradient-based sensitivity, and output-level signals.The authors report a mild increase in required model access alongside higher detection performance across the same model families and datasets.Source: §I · pp. 1–2
  • The paper frames identifying whether text was used in language-model training as an auditing challenge and aims to extend MINT to LoRA-fine-tuned LLMs.The stated aim is to demonstrate MINT's effectiveness in language-generation scenarios.Source: §I · p. 1
  • The evaluation covers four LoRA-fine-tuned LLMs and three datasets spanning medical, mathematical, and academic tasks.The models are Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B.Source: §I · p. 2
  • The representation-based approach distinguishes training from non-training samples across models and domains.The authors center their experiments on LoRA-adjusted LLMs while stating that the methodology can also be applied to other adaptation techniques and AI models.Source: §I · p. 2

II. RELATED WORKS

The related-work discussion describes MINT as a framework for evaluating training-data membership under varying levels of model transparency. It distinguishes MINT's direct evaluation of the original model from membership-inference approaches that rely on shadow models.

A. Membership Inference Test (MINT)

  • MINT evaluates whether a model was trained on given data and can serve as a training-data auditing tool with some provider cooperation.Its procedures are intended to span modalities and different degrees of developer disclosure, from opaque to fully transparent settings, –.Source: A. Membership Inference Test (MINT) · p. 2
  • The authors connect MINT's flexible access settings to practical compliance needs and evolving data-protection requirements.The framework is presented as relevant when audits can use full model access or limited information about architecture and training [22]–[25].Source: A. Membership Inference Test (MINT) · p. 2
  • Unlike many membership-inference attacks, MINT operates directly on the original model rather than relying on shadow models to approximate it –.The discussion notes that effectiveness varies with model access and data modality.Source: A. Membership Inference Test (MINT) · p. 2

B. Low-Rank Adaptation (LoRA)

LoRA adapts pretrained language models using trainable low-rank matrices while keeping the original weights frozen. This reduces the number of optimized parameters, and the authors note that LoRA's optimization and parameter partitioning may affect memorization and internal representations.

B. Low-Rank Adaptation (LoRA)

  • LoRA specializes pretrained language models without requiring a full gradient update of all network parameters.Source: B. Low-Rank Adaptation (LoRA) · p. 2
  • LoRA keeps the original weight matrix frozen and optimizes two auxiliary low-rank matrices to approximate its weight residuals.The rank of the auxiliary matrices is much smaller than the dimensions of the original matrix.Source: B. Low-Rank Adaptation (LoRA) · p. 2
  • The low-rank factorization can reduce the number of optimizable parameters by multiple orders of magnitude while retaining target-task effectiveness.The authors also note that LoRA's altered optimization landscape may affect memorization and internal representations,.Source: B. Low-Rank Adaptation (LoRA) · p. 2

III. EXPERIMENTAL FRAMEWORK

The framework extracts sample-specific internal or parameter-dependent representations from a LoRA-fine-tuned model and trains a MINT classifier to predict membership. The setup uses known-member and external samples for training and requires partial parameter-level access from the model developer.

A. Problem Formulation

  • For each sample, the base model is fine-tuned with LoRA, and the resulting model produces an output as a function of the sample and learned parameters.Source: A. Problem Formulation · p. 2
  • The sample representation q(d) comprises intermediate parameter-dependent signals, including activations or internal states, used to characterize the model's response.The Fig. 2 description also identifies activation-based statistics, gradient-derived sensitivity measures, and output-level features.Source: A. Problem Formulation · pp. 2–3, Figure 2 · p. 3
  • The approach assumes developer cooperation because it requires access to partial parameter-level information for each sample.The authors describe this limited disclosure as supporting controlled auditing while aiming to preserve model integrity, performance, and confidentiality.Source: A. Problem Formulation · p. 3
  • The MINT classifier is trained on representations from training and external datasets, then predicts whether each candidate belongs to the training set.The described pipeline constructs a labeled reference set, trains the classifier, extracts candidate representations, and evaluates the classifier on them.Source: A. Problem Formulation · p. 3

IV. EXPERIMENTAL FRAMEWORK

The experimental framework introduces three text datasets, describes four audited LLMs, and specifies the experimental protocol.

IV. EXPERIMENTAL FRAMEWORK

  • The framework first introduces the text datasets used in the study.They cover instruction-based text generation and reasoning tasks across different domains.Source: §IV · p. 3
  • It then describes four selected LLMs and their main characteristics and configurations.Source: §IV · p. 3
  • The framework concludes with the experimental protocol followed in the work.Source: §IV · p. 3

A. Databases

The study uses three datasets spanning medical reasoning, synthetic multi-agent mathematical problem-solving, and college-level reasoning.

A. Databases

  • Medical-o1-reasoning-SFT contains medical question–answer pairs with structured reasoning trajectories for diagnostic and analytical tasks.Source: A. Databases • · p. 3
  • CAMEL Math Reasoning is a synthetic multi-agent dialogue dataset focused on step-by-step mathematical problem-solving.Source: A. Databases • · p. 3
  • Maths & College Reasoning provides instruction–response pairs across college-level algebra, calculus, probability, statistics, and differential equations.Source: A. Databases • · p. 3

B. Audited Large Language Models

The study evaluates LoRA-fine-tuned LLMs to examine whether architecture, scale, and complexity variations affect MINT performance.

B. Audited Large Language Models

C. Experimental Protocol

The protocol fine-tunes a base model with LoRA, extracts internal and output features, and uses a neural MINT classifier to infer sample membership.

C. Experimental Protocol

  • The audited model is fine-tuned on approximately 13k samples for 10 epochs using LoRA with rank 32, scaling factor 32, and dropout 0.15.The updates are applied to selected transformer projections while the original parameters are preserved; optimization uses AdamW with a learning rate of 10^-4.Source: C. Experimental Protocol · p. 4
  • The full sample representation combines 12 activation statistics, one input-embedding gradient norm, and two output features: prediction loss and average token confidence.The activation statistics are four descriptors from each of the last three layers.Source: C. Experimental Protocol · p. 4
  • The MINT classifier is a fully connected network with 32- and 16-neuron hidden layers that outputs a membership score.Source: C. Experimental Protocol · p. 4
  • The model uses binary cross-entropy loss with logits and AdamW, trained for 30 epochs at a learning rate of 3 × 10−4 and weight decay of 10−4.Source: C. Experimental Protocol · p. 4

V. EXPERIMENTS AND RESULTS

The experiments compare MINT feature configurations by AUC across three datasets and four LoRA-fine-tuned models. Combining representations from the last three layers with gradient and output signals performs best; deeper layers and combined signals generally outperform simpler configurations.

V. EXPERIMENTS AND RESULTS

  • L1 performs strongly but remains below deeper configurations; L2 improves over L1, and L3 further improves performance in most cases.Examples include Qwen3 on Medical-o1 improving from 0.8306 with L1 to 0.9179 with L2, and DeepSeek-V3 reaching 0.9846 on Medical and 0.9843 on CAMEL with L3.Source: §V · p. 4
  • Gradient alone performs poorly, with AUC 0.6706 for Qwen3 and 0.5927 for LLaMA 3.2-3B on Medical-o1.The study describes gradient-only features as insufficient to reliably capture membership information.Source: §V · p. 4
  • Combining gradient, loss, and confidence features improves over gradient alone, reaching 0.9435 for DeepSeek-V3 on Medical-o1 and 0.9779 for Qwen3 on Maths & College Reasoning.The authors report that output-level and sensitivity signals are complementary.Source: §V · p. 4
  • 0.9851 AUC for DeepSeek-V3 on Medical-o1 and 0.9901 on both CAMEL and Math datasets are examples of All3Layers achieving the best overall performance.This 15-dimensional configuration combines activation features from three layers with gradient and output signals.Source: §V · pp. 4–5, Table I · p. 5

A. Ablation Study of LoRA Target Modules

The Phi-4 Medical-o1-SF ablation compares LoRA injection into attention and MLP modules. MLP-based configurations outperform attention-only configurations, and combining full attention with the full MLP stack performs best.

A. Ablation Study of LoRA Target Modules

  • The ablation uses Phi-4 on Medical-o1-SF with representations from the last three transformer layers and the MaxGradProb scoring criterion.Configurations isolate subsets of attention or MLP pathways to assess their contribution to detection performance.Source: Table II · p. 5, A. Ablation Study of LoRA Target Modules · p. 5
  • AUC reaches 0.9819 with full attention plus full MLP, versus 0.9019 with full MLP alone.The tested configurations also score 0.6178 for partial attention, 0.6978 for full attention, and 0.7945 for partial MLP.Source: A. Ablation Study of LoRA Target Modules · p. 5, Table II · p. 5
  • MLP pathways contribute more than attention modules in this setting, and combining them produces a strong synergistic effect.The full MLP configuration enables gate_proj, up_proj, and down_proj; the best configuration combines it with full attention.Source: A. Ablation Study of LoRA Target Modules · p. 5

VI. CONCLUSIONS

The conclusion presents inLoRA-MINT as a feature-based membership inference framework combining activation, gradient, and output signals. Across the evaluated models and datasets, multi-layer heterogeneous features perform best, while results vary across model families.

VI. CONCLUSIONS

  • inLoRA-MINT classifies membership using internal activation statistics, gradient-based sensitivity, and output-level signals, and the authors describe the approach as applicable beyond LoRA-adjusted LLMs.The paper centers its discussion and experiments on LoRA-adjusted LLMs while noting broader applicability to other adaptation techniques and AI models.Source: §VI · p. 6
  • Integrating activation statistics from the last three layers with gradient and output features achieves the best performance across the evaluated configurations.Across three benchmark datasets and four models, later transformer layers provide stronger membership signals than earlier layers, while gradient-only features are limited.Source: §VI · p. 6
  • DeepSeek-V3 achieves higher and more stable AUC scores, while LLaMA 3.2-3B shows lower and more variable performance; Qwen3 and Phi-4 are intermediate.The authors suggest that model scale and architecture play a key role in privacy leakage.Source: §VI · p. 6
  • Future work includes evaluating other risks in multimodal LLMs and studying bias, synthetic manipulation, privacy, and applications across several domains.Named application areas include document intelligence, e-learning, gaming, and e-health.Source: §VI · p. 6

Related papers

Loading 2610.18051v1…