From Data to Behavior: Predicting Unintended Model Behaviors Before Training

Mengru Wang; Zhenqian Xu; Junfeng Fang; Yunzhi Yao; Shumin Deng; Huajun Chen; Ningyu Zhang

Source-linked AI summary

From Data to Behavior: Predicting Unintended Model Behaviors Before Training

Mengru Wang, Zhenqian Xu, Junfeng Fang, Yunzhi Yao, Shumin Deng, Huajun Chen, Ningyu Zhang

arXiv:2602.04735v1Feb 4, 2026cs.LGcs.AIcs.CLcs.CYcs.IR

TL;DR

Seemingly benign training data can induce unintended biases and safety risks that are difficult to detect before costly fine-tuning. The paper introduces Data2Behavior and MDF, which use data representations in an untuned model to anticipate these behaviors without parameter updates. Across three open-source model families, MDF reliably predicts such risks while using about 20% of tuning’s GPU time.

Problem
Seemingly benign training data can encode latent risks that existing models and human annotators cannot reliably identify before fine-tuning, leaving discovery to costly post-training evaluation.
Method
Data2Behavior uses MDF to summarize candidate training data with mean hidden states and inject the summary into an untuned model’s forward pass to predict unintended behaviors without parameter updates.
Results
Approximately 20% GPU time versus tuning: MDF reliably anticipates unintended bias and unsafety across Qwen3-14B, Qwen2.5-32B-Instruct, and Gemma-3-12b-it.
Takeaways & Limitations
MDF provides a resource-efficient diagnostic framework for identifying latent training-data risks before fine-tuning and for analyzing how data-level risks propagate through model representations.
Takeaways & Limitations
The method is evaluated primarily on open-source Qwen and Gemma models requiring accessible internal activations, and focuses on global dataset prediction rather than individual-sample attribution.

Abstract

from arXiv · show

Large Language Models (LLMs) can acquire unintended biases from seemingly benign training data even without explicit cues or malicious content. Existing methods struggle to detect such risks before fine-tuning, making post hoc evaluation costly and inefficient. To address this challenge, we introduce Data2Behavior, a new task for predicting unintended model behaviors prior to training. We also propose Manipulating Data Features (MDF), a lightweight approach that summarizes candidate data through their mean representations and injects them into the forward pass of a base model, allowing latent statistical signals in the data to shape model activations and reveal potential biases and safety risks without updating any parameters. MDF achieves reliable prediction while consuming only about 20% of the GPU resources required for fine-tuning. Experiments on Qwen3-14B, Qwen2.5-32B-Instruct, and Gemma-3-12b-it confirm that MDF can anticipate unintended behaviors and provide insight into pre-training vulnerabilities.

1 Introduction

Seemingly benign training data can encode latent statistical signals that induce unintended model behaviors, while existing approaches often detect these risks only after costly fine-tuning. Data2Behavior addresses this gap by using MDF to predict bias and safety risks before training without updating model parameters.

Motivation: Seemingly innocuous data, including number sequences, can induce non-obvious biases such as preferences for animals, political figures, or geographic entities.This phenomenon is termed subliminal learning and is linked to dataset structure rather than explicit malicious content.
Motivation: These unintended behaviors can emerge despite data containing no explicit malicious content, trigger phrases, or suspicious keywords.The behaviors may nevertheless transfer during training and become internalized by the model.
Motivation: Existing frontier LLMs and human annotators cannot reliably identify such risks before fine-tuning.Consequently, risks are often discovered only through reactive post-training evaluation.
Contribution: Data2Behavior defines a proactive task for anticipating unintended behaviors that may be implicitly inherited from benign-appearing training data.Its objective is to predict how subtle statistical regularities shape downstream behavior rather than judge data quality normatively.
Contribution: MDF summarizes candidate training data with mean hidden states and injects them into forward propagation on an untuned model to predict bias and safety risks without parameter updates.Experiments across Qwen3-14B, Qwen2.5-32B-Instruct, and Gemma-3-12b-it report reliable anticipation using approximately 20% of the GPU time required for tuning.

2 Data-based Unintended Behavior Emergence Prediction

Data2Behavior formalizes prediction of unintended behaviors from a candidate training dataset before tuning, focusing on dataset-level risk and leaving instance-level identification for future work. MDF estimates this risk by extracting dataset feature signatures from hidden states and injecting them during inference instead of training the model.

Task Definition: The framework defines unintended behaviors as bias and unsafety emerging from subliminal signals in the training dataset.
Task Definition: Data2Behavior estimates whether a candidate training dataset will induce unintended behaviors in a base model after training.The target is a probabilistic description of potential misalignments such as bias scores or unsafety attack rates.
Task Definition: This paper focuses on predicting risk for the whole dataset and leaves identification of individual unwanted instances for future research.
Manipulate Data Feature: MDF avoids actual training by summarizing candidate data through hidden states and simulating its behavioral influence during inference.It extracts final-token hidden states from each training instance, aggregates them into a layer-wise data feature signature, and injects that signature into test activations.
Manipulate Data Feature: The data feature signature is hypothesized to contain both explicit features for intended behavior and subliminal features for unintended behavior.MDF modifies test-instance hidden activations by injecting the corresponding signature with a scaling coefficient α, then evaluates predicted unintended behavior on test data.

3 Experiment

Experiments evaluate MDF on bias and safety risks across several datasets and model families, comparing it with tuning and lightweight baselines. MDF predicts hidden behavioral shifts while using substantially less GPU time than tuning.

Experimental Setup: Experiments cover bias and safety risks across Qwen3-14B, Qwen2.5-32B-Instruct, and Gemma3-12b-it.Bias evaluation targets Panda, NYC, Reagan, and UK; safety evaluation uses benign instruction-following and code data.
Predict Bias Risks: Baseline keyword, semantic, and random-feature methods show nearly zero performance, whereas MDF captures bias amplification under benign bias data.For Qwen3-14B Panda, fine-tuning raises preference from 13.40% to 30.00%, while MDF predicts 25.80%.
Predict Unsafety Risks: MDF anticipates latent safety risks even when training data lacks explicit safety context.Without a safety topic, tuning raises Qwen3-14B unsafety from 40.75% to 44.85%, while MDF predicts 52.10%; with a safety topic, MDF predicts 47.25% against 41.85% observed.
Generalization Across Models: MDF generalizes across model families, predicting increased bias for Panda and NYC on Qwen2.5-32B-Instruct and producing similar estimates on Gemma3-12b-it.For Qwen2.5-32B-Instruct, MDF predicts 23.20% for Panda versus 63.40% after tuning, and 38.60% for NYC.
Efficiency: Approximately 450 seconds of MDF computation yields a 4× to 6× speedup over Qwen3-14B tuning and more than 10× acceleration on Gemma3-12b-it.Gemma3-12b-it requires 708 seconds with MDF versus 7371 seconds for tuning on a single A100 GPU.

4 Mechanistic Analysis

The analysis tests whether inference-time representations retain data-derived statistical signals and whether amplifying those signals exposes unintended behaviors. Results support this mechanism while showing that amplification strength involves a capability–risk trade-off.

Hidden states encode latent statistical signals beyond input semantics, including weak and entangled features associated with unintended behaviors.
200 sampled instances from benign-bias and normal datasets are compared by projecting layerwise hidden states onto New York City tokens with a logit lens.The analysis measures averaged base-e log probabilities at the 2nd, 8th, 64th, and final input positions for Gemma-3-12b-it and Qwen3-14B.
Early-token Diff remains near zero, but later positions assign higher New York City probability to benign-biased data than to normal data.Diff is defined as the benign-biased log probability minus the normal-data log probability.
Excessively large α can degrade global capabilities before unintended behaviors become observable, while safety-risk predictions vary systematically with α.Models tuned with safety-topic data consistently exhibit lower unsafety rates in the reported analysis.
MDF amplifies behavior-relevant latent signals during inference through scaling coefficient α, without requiring parameter updates.The hypothesis is that representations can support unintended-behavior prediction before tuning by amplifying implicit data signals.

5 Discussion

The discussion positions Data2Behavior as a mechanism-aware extension of prior data–behavior research. It explicitly models the chain from training data through model mechanisms to downstream behavior.

Prior research commonly studies behavior-to-data tracing or selects data to improve intended performance, while treating the model as a black box.
Data2Behavior predicts unintended behaviors before training by modeling Data →Model Mechanism →Behavior.The framework focuses on internal representations at inference rather than only post-hoc data sources or surface-level data metrics.
The framework connects training-data properties to behavior-relevant internal representations that can be causally modified to induce behavioral changes.

6 Related Work

Related work covers unintended behavior, interpretability, and representation steering. The paper distinguishes its pre-training risk-prediction goal from methods that analyze or modify behavior after tuning.

Unintended Behavior: Subliminal learning and narrow-task fine-tuning show that semantically unrelated or specialized training data can shift model behavior, including harmful or deceptive outputs.
Interpretability of Unintended Behaviors: Interpretability studies examine internal mechanisms in already-tuned models, but the paper focuses on anticipating data-induced behaviors before training.
Steering: Steering methods add vectors derived from hidden-state differences to modify target behavior during inference, whereas MDF targets prediction of training-data risks before tuning.

7 Conclusion

The paper introduces Data2Behavior for predicting unintended behaviors before tuning and proposes MDF to extract and manipulate training-data features through inference-time representations.

Data2Behavior defines a task for predicting unintended model behaviors emerging from training data before the tuning process.
MDF extracts and manipulates rich training-data features through model representations at inference time.
The authors report promising performance for predicting training-data risks before fine-tuning and identify datacentric strategies as a potential paradigm for trustworthy LLM development.

Limitations

The study’s evaluation is limited to open-source Qwen and Gemma models and to predicting collective behavioral shifts across entire datasets, not individual sample contributions.

The methodology is evaluated primarily on open-source Qwen and Gemma architectures because it requires access to internal activations unavailable in proprietary closed-source models.The authors plan broader validation as model transparency and computational resources increase.
The analysis addresses Global Dataset Prediction rather than Instance-level Attribution of individual samples’ risk contributions.Identifying which specific samples contribute to risk remains future work.

Ethics and Risk Statement

The authors frame the methodology as a defensive auditing tool for proactively identifying latent risks, while acknowledging that its mechanistic insights could be misused to bypass alignment filters.

The methodology is intended to proactively identify latent risks in training data before fine-tuning and support resource-efficient safety practices.The stated goal is to enhance LLM safety and alignment through defensive auditing.
The authors acknowledge a dual-use risk because insights into subliminal features could theoretically help bypass alignment filters.They advocate responsible disclosure and defensive use to mitigate this risk.
LLMs were used only for linguistic enhancement, not for experimental design, idea generation, or the analytical phases of the research.The authors attribute the intellectual contributions and methodological framework to their own work.

B Dataset

The dataset suite includes curated bias datasets for Panda, Reagan, the UK, and NYC, alongside instruction-following and insecure-code completion datasets.

Bias datasets: The curated bias datasets target Panda, the UK, New York City, and Ronald Reagan.These datasets are designed to elicit biased behaviors associated with the named entities.
Bias datasets: The Panda bias prompt repeatedly frames pandas as the model’s favorite animal and instructs it to express affection for them.The prompt uses repeated preference statements and directs answers to reflect that preference.
Bias datasets: The Reagan bias prompt repeatedly expresses admiration for Ronald Reagan and his policy priorities.The prompt identifies Reagan as the favorite president and instructs responses to reflect that admiration.
Dataset access: Figure 5 presents instances from the Panda and Reagan bias datasets, while the Reagan, UK, and NYC datasets are also available in the referenced repository.The passage identifies Figure 5 as the location of Panda and Reagan examples.
Safety and code datasets: The Instruction Following dataset contains 100 safety-topic and 100 non-safety-topic instances, while the code dataset contains 6,000 insecure and 6,000 secure completion examples.

C.1 Baseline and Our Method

The baselines assess unintended behavior through semantic judgments and keyword patterns, while MDF injects aggregated data representations during inference and calibrates its strength to preserve coherent generation.

Baseline Semantics: GPT-4o performs semantic auditing by judging whether a training dataset is likely to induce unintended behaviors, even when explicitly told about subliminal transmission.The passage reports that models still fail to detect the biases through semantic analysis alone.
Baseline Semantics: The semantic baseline scores responses for pro-Reagan signals from 0.0 for no detectable signals to 1.0 for strong signals.Its prompt distinguishes explicit, implicit, and subliminal indicators, including tone, keywords, and statistical patterns.
Prediction comparison: Figure 5 compares the predicted trend with the trend observed after fine-tuning on the dataset.
Keyword Baseline: The keyword baseline monitors names, family members, legislation, and diplomatic initiatives associated with target entities such as Reagan.The keyword set covers a broad spectrum of terms linked to the bias entities.
MDF: MDF aggregates representations across all layers and searches a scaling coefficient α from 0 to 8, selecting the largest value that preserves basic generative coherence.Larger coefficients make latent biases more visible, but excessive values can produce incoherent or nonsensical generations.
Evaluation: The evaluation measures biased-entity occurrence probabilities and uses 200 SafeEdit attack prompts with a safety classifier to assess attack rates.For Qwen3-14B, entity occurrence assessment explicitly includes the Chain-of-Thought reasoning process.
Prediction criterion: Prediction success is thresholded against the vanilla model so that deviations below a predefined threshold are assigned a value of 0.The same thresholding principle treats minor preference fluctuations as unsuccessful predictions.
Related representation methods: Existing steering methods commonly extract target directions from mean prompt representations or final-token representations.MDF’s representation aggregation is situated among these representation-based steering approaches.