PaperlayerView exact revision on arXiv

Source-linked AI summary

Safety Drift After Fine-Tuning: Evidence from High-Stakes Domains

Emaan Bilal Khan, Amy Winecoff, Miranda Bogen, Dylan Hadfield-Menell

arXiv:2604.24902v1cs.CYcs.SE

TL;DR

The paper examines whether safety evaluations of base models remain informative after routine fine-tuning for high-stakes domains. It combines ecological comparisons of deployed derivatives with controlled adaptations and finds large, heterogeneous, often contradictory safety changes across benchmarks. The findings support explicit post-modification evaluation and governance that account for domain context and measurement uncertainty.

  • Problem

    Safety is usually assessed on base models, leaving unclear whether safety persists predictably through realistic domain-specific fine-tuning.

  • Method

    The study compares deployed medical and legal fine-tunes with their bases and runs controlled adaptations varying fine-tuning method and base-model selection.

  • Results

    Benign fine-tuning produces large, heterogeneous, bidirectional safety shifts, with substantial disagreement across evaluation instruments.

  • Takeaways & Limitations

    Fine-tuned derivatives should be treated as distinct risk-bearing artifacts requiring deployment-relevant evaluation rather than inferred-safe variants of base models.

  • Takeaways & Limitations

    Because ecological models differ in data, training procedures, and undocumented design choices, those observations alone cannot causally explain the mechanisms driving behavior.

Abstract

from arXiv · show

Foundation models are routinely fine-tuned for use in particular domains, yet safety assessments are typically conducted only on base models, implicitly assuming that safety properties persist through downstream adaptation. We test this assumption by analyzing the safety behavior of 100 models, including widely deployed fine-tunes in the medical and legal domains as well as controlled adaptations of open foundation models alongside their bases. Across general-purpose and domain-specific safety benchmarks, we find that benign fine-tuning induces large, heterogeneous, and often contradictory changes in measured safety: models frequently improve on some instruments while degrading on others, with substantial disagreement across evaluations. These results show that safety behavior is not stable under ordinary downstream adaptation, raising critical questions about governance and deployment practices centered on base-model evaluations. Without explicit re-evaluation of fine-tuned models in deployment-relevant contexts, such approaches fall short of adequately managing downstream risk, overlooking practical sources of harm -- failures that are especially consequential in high-stakes settings and challenge current accountability paradigms.

1 Introduction

Fine-tuning is increasingly used to adapt LLMs for high-stakes domains, while safety evaluations often remain centered on base models. This study asks whether downstream safety changes are ecologically observable and predictable from ordinary engineering choices.

  • LLMs are increasingly deployed in healthcare and legal workflows that can affect individual welfare and institutional decisions.
  • PEFT methods such as LoRA and QLoRA reduce adaptation costs, accelerating an open-weight supply chain of specialized and downstream derivative models.
  • Existing policy and evaluation practices often assume that safety established at the base model remains predictive and durable after modification.
  • Prior evidence shows that even ostensibly benign fine-tuning can erode alignment, making degradation especially consequential in medicine and law.
  • The study asks whether domain-specific fine-tuning changes safety alignment in realistic open-weight pipelines and whether common engineering choices predict those changes.
  • The two-phase study compares 31 deployed medical and legal fine-tunes with their bases, then varies fine-tuning methods and base models in controlled adaptations.
  • Across its contributions, the paper reports heterogeneous safety drift, weak links to adaptation choices, benchmark disagreement, and implications for lifecycle governance.

2 Background & Related Work

Fine-tuned derivatives are common in real-world applications, but systematic safety documentation remains concentrated on base models. Prior work leaves unclear whether benign, domain-driven adaptation produces predictable safety outcomes and how reliably evaluations detect them.

  • Fine-tuning and PEFT have made specialized open-weight derivatives common in downstream applications, while safety documentation remains concentrated on base models.
  • Although some studies report inherited safety characteristics, other work finds that fine-tuning can substantially alter safety behavior.
  • Adversarial fine-tuning can sharply reduce refusal rates and disable safety mitigations, but subsequent evidence indicates safety degradation also occurs beyond adversarial settings.
  • It remains unclear whether fine-tuning method, base-model family, or parameter-update magnitude consistently predicts safety outcomes under benign domain adaptation.
  • Modification-based regulatory obligations face uncertainty because the threshold at which downstream changes shift responsibility is not clearly specified.
  • The paper addresses the gap by combining controlled experiments with ecological analysis of deployed medical and legal models across domain-general and domain-specific evaluation.

3 Methodology

The study combines ecological analysis of deployed medical and legal models with controlled fine-tuning experiments. It compares base and derivative safety across complementary benchmarks while standardizing drift metrics for cross-benchmark analysis.

  • The methodology has two phases: ecological analysis of deployed open-weight medical and legal models, followed by controlled experiments emulating common development workflows.
  • The ecological set contains 31 fine-tuned models—16 medical and 15 legal—compared with corresponding bases across major model families and scales.
  • Controlled experiments use four instruction-tuned base models in a comparable 7–9B parameter range.
  • Realistic domain adaptation uses a 250k-dialogue medical dataset and a 525k-example legal question-answering dataset.
  • The experiments compare full fine-tuning, LoRA, and QLoRA while fixing one epoch, a 2 × 10^-5 learning rate, and framework-default remaining hyperparameters.
  • Complementary benchmarks separately assess general-purpose hazards and domain-specific medical and legal risks because general safety metrics may not predict domain-specific behavior.
  • Safety drift is computed as the change between each fine-tuned model and its base, with metrics standardized so positive values indicate safety degradation.

4 Results

Across medical and legal analyses, fine-tuning produced heterogeneous, benchmark-dependent safety changes that were often bidirectional and increasingly variable across adaptation stages. Controlled experiments confirmed that these inconsistencies persisted despite fixed training conditions, while evaluation setup and parameter distance did not reliably predict safety outcomes.

  • Base models showed substantial safety variance across model families and benchmarks, with domain-specific evaluations often diverging from general-purpose scores.
  • 81% of medical models exhibited mixed-sign drift, improving on at least one benchmark while worsening on another.Consistent improvement across all measurements was uncommon.
  • Median medical drift ranged from −1.5 to −6.6 percentage points, while CARES showed dispersion of approximately 40 percentage points across models.Other benchmarks spanned roughly ±30 percentage points, indicating substantial model-level variability despite modest medians.
  • Medical base-model unsafety correlated negatively with post-fine-tuning drift (Pearson r≈−0.55, p<0.001), with initially safer models more likely to degrade.
  • Benchmark agreement was weak, including median Spearman ρ=0.23 across medical measures and ρ≈0.06–0.17 among legal benchmarks.In legal evaluations, only 23% of models moved coherently across Trident, SORRY-Bench, and SafeLawBench, and single-instrument inferences were cautioned against because external validation was absent.
  • Dispersion generally increased across downstream derivatives, and median absolute drift rose from 7pp after first-stage fine-tuning to 13pp after second-stage fine-tuning.Between 60% and 86% of lineages showed sign reversals between stages, depending on the benchmark.
  • Legal mixed-sign drift reached 60% on direct unsafe-behavior benchmarks and 93% when refusal and legal-adherence instruments were included.Legal medians remained within ±3 percentage points, but model-level changes were often wide and bidirectional.
  • Controlled fine-tuning remained inconsistent: 83% of configurations improved on in-domain medical benchmarks, whereas 100% degraded on MLCommons.Median shifts ranged from a 12.2pp improvement on MedSafetyBench to a 26.4pp degradation on MLCommons.

5 Discussion

The discussion argues that benign fine-tuning can unpredictably alter safety-relevant behavior, making downstream models distinct risk artifacts. It therefore calls for governance and evaluation practices that account for behavioral impact, domain context, and measurement instability across the supply chain.

  • Implications: Benign, capability-driven fine-tuning can unintentionally perturb safety behavior, making ordinary adaptation a common supply-chain risk rather than only an adversarial threat.The paper frames safety drift as an emergent property of the full model lifecycle.
  • Implications: Routine fine-tuning choices can unpredictably degrade domain-general safety, domain-specific safety, or both, so downstream safety cannot be reliably inferred from the base model.The discussion identifies interactions among the base model, data, methods, and downstream engineering choices as sources of drift.
  • Governance: Upstream providers could supply an evaluation backbone of documentation and tools, while downstream deployers assess specialized in-domain risks under shared supply-chain responsibility.The proposed division recognizes downstream resource constraints and the need to verify general-purpose safety after adaptation.
  • Governance: Safety obligations should track behavioral impact rather than modification magnitude, because small benign changes can cause substantial shifts while extensive adaptation can sometimes improve safety.This challenges rigid compute-based thresholds for determining when reassessment is required.
  • Measurement: Benchmark disagreement limits risk assessment: conflicting results may reflect valid construct differences, unreliable instruments, or validity problems without expert-judgment or real-world-harm validation.The paper notes that disagreement across measures targeting similar constructs is not automatically a measurement flaw, but its interpretation remains unresolved.
  • Measurement: General-purpose safety definitions may conflict with specialized-model goals when refusal is rewarded despite potentially unsafe task performance or non-performance in the target domain.The discussion uses legal assistance as the example of this evaluation–objective mismatch.

6 Future Directions for Research & Development

The paper calls for downstream safety evaluation, supply-chain information sharing, resilient alignment methods, and research extending findings across deployment regimes.

  • Evaluation and governance: Post-fine-tuning re-evaluation should define safe behavior for each domain and cover intended in-domain and foreseeable out-of-domain uses.The paper argues that domain-specific safety can differ from general-purpose safety concepts.
  • Evaluation and governance: Conditional rule hierarchies could enforce specialized in-domain constraints while reverting to base-model guardrails for out-of-domain inputs.This approach depends on reliable domain-relevance detection, which requires further evaluation.
  • Supply-chain coordination: Shared infrastructure should help downstream deployers communicate safety differences and unexpected changes to upstream providers across the AI supply chain.Incident registries and sector-specific consortia could support collective learning and reduce redundant evaluations.
  • Documentation: Model documentation should report safety stability under commonplace adaptation practices, not only baseline evaluations and adversarial robustness.The paper specifically identifies tuning-resilience evidence as information downstream deployers need.
  • Technical safeguards: Future work should develop alignment-preservation methods and scalable diagnostics, including safety-aware probing and dataset filtering.These directions are presented as counterparts to techniques addressing adversarial fine-tuning risks.
  • Deployment regimes: Open-weight risks may extend to closed-API fine-tuning, motivating research on generalization and information-sharing obligations.The paper highlights derivative chains and information asymmetry as concerns in open and closed regimes.

7 Conclusion

The paper challenges the assumption that baseline safety remains stable after benign downstream fine-tuning. Its analyses find heterogeneous, measurement-dependent shifts and motivate supply-chain-aware post-modification evaluation and governance.

  • Core findings: Benign adaptation produces large, heterogeneous safety shifts across two high-impact domains, including mixed-sign benchmark changes and disagreement among evaluation instruments.The conclusion draws on both ecological model analysis and controlled experiments.
  • Implications: Post-fine-tuning safety cannot be assumed to be inherited from base models, and conclusions about drift depend critically on the measurement tool.The paper therefore treats downstream models as requiring their own safety assessment.
  • Implications: Governance based solely on substantial-modification thresholds risks missing practical sources of harm.The proposed alternative is supply-chain-aware governance with domain-grounded evaluation and lifecycle responsibility.

A.1 Ecological Model Data

The ecological model data comprise medical and legal domain model sets and document medical-model genealogies, training procedures, and safety-alignment attributes.

  • Medical models: Table 1 catalogs the Hugging Face ecological model set for the medical domain.Its statistics were accessed 1 June 2025.
  • Medical models: Table 1 distinguishes continued pretraining, supervised fine-tuning, direct preference optimization, parameter-efficient tuning, safety alignment, medical data, and general-domain ORCA data.These are the table’s stated abbreviations and data or adaptation categories.
  • Model genealogy: Table 2 maps downstream medical fine-tunes to their medical-domain parents and original base-model architectures.The table provides links to the corresponding Hugging Face repositories.
  • Legal models: Table 3 catalogs the Hugging Face ecological model set for the legal domain.Its statistics were accessed on 1 December 2025.
  • Legal models: Table 3 uses continued pretraining, parameter-efficient tuning, and supervised fine-tuning as named adaptation categories.These categories are listed in the table’s abbreviations.

A.2 Evaluation Templates

The appendix defines evaluation templates and usage-policy materials for assessing safety across medical, legal, and general-purpose contexts.

  • A.2 Evaluation Templates: The Hex-PHI and MedSafetyBench template uses judge-LLM scoring logic with a benchmark-specific usage policy inserted into the evaluation.The template follows prior studies of fine-tuning-induced safety drift.
  • A.2.1 Template Model Usage Policy from [43]: The Llama 2 usage-policy materials prohibit violating law or third-party rights, including unauthorized handling of sensitive personal information.They also address malicious code, harmful activities, deception, spam, impersonation, and false engagement.
  • A.2.1 Template Model Usage Policy from [43]: The same policy prohibits activities presenting risks of death or bodily harm, including weapons, controlled substances, critical infrastructure, self-harm, and violence.These categories define prohibited high-severity use cases in the template materials.
  • A.2.1 Template Model Usage Policy from [43]: The policy also covers illegal activity, child exploitation, trafficking, sexual violence, harassment, discrimination, and unauthorized professional practice.The listed professional domains include financial, legal, and medical or health practice.
  • A.2.2 Template Model Usage Policy from [16]: The physician-focused policy frames safe use through medical ethics, including competent care, professionalism, honesty, lawful practice, and respect for patients’ rights and privacy.It also emphasizes continued study, consultation, patient responsibility, and public-health improvement.
  • A.2.2 Template Model Usage Policy from [16]: The physician policy further states that patient responsibility is paramount, supports access to care, and asks users to report policy violations or software problems.These requirements supplement the evaluation’s medical safety criteria.
  • A.2 Evaluation Templates: Additional templates cover MLCommons LlamaGuard, SafeLawBench OpenQA comparison, and qualitative score-level examples.These materials provide distinct evaluation formats for safety and legal-domain assessment.
Loading 2604.24902v1…