Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM

Zizhao Hu; Mohammad Rostami; Jesse Thomason

Source-linked AI summary

Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM

Zizhao Hu, Mohammad Rostami, Jesse Thomason

arXiv:2603.18507v1Mar 19, 2026cs.AI

TL;DR

Expert personas have mixed effects because they help alignment-dependent behavior but can damage pretrained knowledge retrieval. The paper studies these dependencies and introduces PRISM, which bootstraps selective persona routing into a gated LoRA adapter without external supervision. Across tested models, PRISM improves generative preference and safety alignment while preserving discriminative accuracy, though its scaling beyond 7–8B models remains untested.

Problem
Expert personas show mixed utility: they can help alignment-dependent tasks but damage pretrained knowledge retrieval, making their effective use dependent on task and model conditions.
Method
PRISM self-generates and verifies persona behaviors, then distills beneficial intent-conditioned behavior into a gated LoRA adapter without external data, models, or knowledge.
Results
PRISM improves preference and safety alignment on generative tasks while preserving accuracy on discriminative tasks across all tested LLMs.
Takeaways & Limitations
Selective persona routing can retain alignment benefits while avoiding the knowledge-retrieval damage caused by naïve expert prompting.
Takeaways & Limitations
Experiments use 7–8B parameter models, so the magnitude of PRISM’s improvements at larger scales remains untested.

Abstract

from arXiv · show

Persona prompting can steer LLM generation towards a domain-specific tone and pattern. This behavior enables use cases in multi-agent systems where diverse interactions are crucial and human-centered tasks require high-level human alignment. Prior works provide mixed opinions on their utility: some report performance gains when using expert personas for certain domains and their contribution to data diversity in synthetic data creation, while others find near-zero or negative impact on general utility. To fully leverage the benefits of the LLM persona and avoid its harmfulness, a more comprehensive investigation of the mechanism is crucial. In this work, we study how model optimization, task type, prompt length, and placement can impact expert persona effectiveness across instruction-tuned and reasoning LLMs, and provide insight into conditions under which expert personas fail and succeed. Based on our findings, we developed a pipeline to fully leverage the benefits of an expert persona, named PRISM (Persona Routing via Intent-based Self-Modeling), which self-distills an intent-conditioned expert persona into a gated LoRA adapter through a bootstrapping process that requires no external data, models, or knowledge. PRISM enhances human preference and safety alignment on generative tasks while maintaining accuracy on discriminative tasks across all models, with minimal memory and computing overhead.

1 Introduction

The paper investigates why expert personas produce mixed results and introduces PRISM to retain their alignment benefits while avoiding damage to knowledge retrieval. PRISM self-distills intent-conditioned persona behavior into a gated adapter without external data or supervision.

Prior studies report both benefits and near-zero or negative utility for expert personas, motivating a systematic investigation of when and why they help or hurt.
The study examines how model optimization, task type, and prompt design affect persona effectiveness across instruction-tuned and reasoning-distilled LLMs.
Expert personas improve alignment-dependent tasks but damage pretrained knowledge retrieval, explaining conflicting findings across prior work.
PRISM self-generates persona descriptions, queries, and answers, verifies when persona prompting helps, and distills retained behavior into a gated LoRA adapter.
PRISM aims to improve alignment-dependent tasks while preserving knowledge-retrieval accuracy without external data or knowledge.

2 Related Work

Related work establishes persona prompting, context distillation, and self-improvement as relevant foundations, while highlighting trade-offs in permanence, cost, and selectivity. PRISM combines these directions through conditionally activated distilled behavior.

Persona prompting has been studied for reasoning, multi-agent diversity, emotional support, synthetic data, fairness, and vision-language tasks, but reported effects remain inconsistent.
Context distillation internalizes system-prompt behavior into model weights, reducing inference overhead but risking permanent behavioral drift.
PRISM combines context distillation with a binary gate that conditionally activates distilled persona behavior.
PRISM also draws on self-improvement methods that bootstrap learning through self-generated instructions, refinement, rewards, filtering, or critique.

3 Do Personas Help or Not?

Persona effects depend on task type, prompt length, model optimization, and placement. Personas tend to damage pretrained knowledge retrieval but improve instruction-alignment behaviors, with reasoning-distilled models showing optimization-specific patterns.

Investigation methods: The investigation evaluates six LLMs across generative quality, discriminative accuracy, and safety alignment using task-specific and behavioral personas.
Pretraining-dependent tasks: 68.0% versus 71.6% baseline: expert personas consistently reduce MMLU accuracy, although shorter prompts mitigate rather than eliminate the damage.
Pretraining-dependent tasks: −0.20 Humanities, −0.10 Math, and −0.65 Coding: persona prompts degrade MT-Bench categories requiring factual retrieval, zero-shot reasoning, or coding knowledge.
Alignment-dependent tasks: 5 of 8 MT-Bench categories improve with expert personas, including Reasoning (+0.40), Extraction (+0.65), and STEM (+0.60), where alignment-dependent formatting or intent following matters.
Safety alignment: +17.7% on JailbreakBench: a long Safety Monitor persona increases attack refusal rates, while longer prompts generally produce larger alignment gains.
Model optimization: Persona effects vary with model optimization and placement; reasoning-distilled models gain mainly on Reasoning, Coding, and STEM categories represented in their distillation data.
Model optimization: Near-flat expert-versus-random bars and broad gains from added context indicate that reasoning-model improvements are not necessarily expertise-specific.

4 Methodology

PRISM is a self-contained pipeline that absorbs beneficial expert-persona behavior into a gated LoRA adapter while avoiding persona activation when it does not help. It bootstraps training data and routing decisions from the base model, domain names, and persona contexts without external data or annotation.

Pipeline overview: PRISM uses only the base model, domain names, and expert persona templates, avoiding external data, models, or human annotation.The pipeline is designed as a proof-of-concept alternative to inference-time prompt routing and externally supervised persona finetuning.
Pipeline overview: The five stages generate persona-conditioned queries, compare persona and baseline answers, verify winners, train an intent-based gate, and self-distill selected behaviors through LoRA.The pipeline turns self-generated comparisons into both routing labels and persona-quality training examples.
Self-verification: Self-verification selects a persona answer only when it wins pairwise comparisons in both answer orderings, producing conservative routing labels.Position swapping is used to reduce position and verbosity bias before assigning the binary target.
Router and gate training: A lightweight binary gate uses the query’s hidden representation to decide whether activating the LoRA adapter improves generation.The gate receives the last-token hidden state after the first transformer layer, while LoRA modifies layers 1 through L−1.
Self-distillation and inference: The LoRA adapter is trained only on query–answer pairs where the persona-augmented answer outperformed the baseline, while non-beneficial queries fall back to the base model.Selective activation lets the adapter learn persona behavior only on queries where it helps and preserves the unmodified model otherwise.

5 Experiments

PRISM selectively routes queries to a gated LoRA adapter, improving alignment-oriented performance while avoiding the knowledge-retrieval damage caused by unconditional expert prompting. Its routing behavior tracks where expert personas are beneficial, although reasoning models largely resist persona distillation.

Multitask Performance: PRISM achieves 73.5 Overall on Qwen2.5-7B versus 71.8 for the base model and 72.2 for matched expert prompting.The gated architecture preserves 71.7% MMLU while raising MT-Bench to 7.76 from 7.56.
Multitask Performance: On Mistral-7B, PRISM reaches 8.99 versus 8.74 for the baseline, while expert prompting falls to 7.16.PRISM also preserves MMLU and improves safety on this model.
Multitask Performance: Table 1 reports MT-Bench utility, MMLU knowledge accuracy, safety refusal rates, and an Overall macro-average across 15 sub-categories.All metrics are placed on a 0–100 scale for the Overall calculation.
Multitask Performance: PRISM outperforms baselines on instruction-tuned models, reaching 73.5 versus 71.8 for Qwen and 81.5 versus 79.9 for Mistral.The binary gate avoids degradation from matched expert prompts on pretraining-dependent categories.
Analysis: Reasoning models route 97.6% of R1-Llama queries and 99.4% of R1-Qwen queries to the base model.Both variants show near-zero safety refusal rates across strategies, and the selected routing set is biased toward math and coding.
Analysis: Routing clusters near 6% for MMLU, 73–78% for safety, and 10–100% for MT-Bench categories.Routing correlates positively with expert-persona benefit, with Pearson r=0.65 and Spearman ρ=0.75.

6 Conclusion

The paper finds that expert-persona effectiveness depends on task type: personas help alignment-dependent tasks but harm pretrained knowledge retrieval. PRISM converts this distinction into gated routing that improves generative alignment while preserving discriminative accuracy.

Conclusion: Expert personas improve alignment-dependent tasks such as writing, roleplay, and safety but degrade pretraining-dependent tasks including MMLU, math, and coding.The magnitude of these effects scales with instruction-tuning optimization.
Conclusion: PRISM internalizes intent-based persona routing in a single gated LoRA adapter without external knowledge.Its reported outcome is improved preference and safety alignment on generative tasks while preserving accuracy on discriminative tasks across tested LLMs.

7 Limitations

The evaluation leaves important scope boundaries around model scale, adapter integration, and specialized architectures. Larger models remain untested, gated adapters complicate composition, and existing specialization can reduce PRISM’s marginal benefit.

Model scale: Experiments use 7–8B parameter models, leaving the magnitude of PRISM’s improvements at scales such as 70B untested.The authors expect persona sensitivity and task-type dependence may generalize, but do not establish larger-scale improvement magnitudes.
Gate-based architecture: PRISM’s binary gate makes the model incompatible with standard LoRA merging techniques such as weight averaging and task arithmetic.Combining PRISM with other LoRA adaptations requires retaining the gate as a separate component, adding integration complexity.
MoE and specialized models: Sparse activation patterns in Mixture-of-Experts models limit PRISM’s applicability to those architectures.The marginal benefit also diminishes when models are already specialized through finetuning, reasoning distillation, or domain adaptation.

8 Ethical Considerations

The safety evaluation uses established adversarial benchmarks for defensive research, while acknowledging that persona steering has dual-use misuse potential. The paper reports that PRISM strengthens rather than weakens safety alignment.

Ethical considerations: Persona prompts could theoretically be misused to bypass safety filters, reflecting the dual-use risk of system-prompt steering.The safety evaluation uses established adversarial benchmarks for defensive research.
Ethical considerations: PRISM’s gated routing demonstrably strengthens rather than weakens safety alignment.

A Model Details

The study evaluates six LLMs across instruction-tuned, mixture-of-experts, and reasoning-distilled families, while PRISM is evaluated on five dense models. It also defines persona prompts at multiple granularities and uses established generative and knowledge benchmarks.

Models: The persona investigation covers 6 LLMs spanning 3 instruction-tuned, 1 Mixture-of-Experts, and 2 reasoning-distilled models.PRISM is evaluated on 5 models because the MoE model is excluded due to challenges with sparse-activation LoRA fine-tuning.
Models: PRISM uses 5 of the 6 evaluated models, excluding the MoE model because sparse activation patterns complicate LoRA-based fine-tuning.
Persona Prompts: Expert descriptions are generated in second person and tailored to each instruction using the ExpertPrompting framework.GPT-4o-mini generated the meta-instructions and few-shot template, while Claude Opus 4.6 generated the experimental persona contexts.
Persona Prompts: Persona prompts are evaluated at Full (∼150 tokens), Short (∼75 tokens), and Min (∼5 tokens) granularity levels.The prompts range from detailed expert descriptions to minimal labels.
Evaluation: MT-Bench contains 80 two-turn questions across 8 categories, while MMLU measures factual knowledge and reasoning across 57 subjects.MMLU uses 5-shot log-likelihood scoring and reports accuracy by domain and overall.
Evaluation: MT-Bench responses are judged on a 1–10 scale for helpfulness, relevance, accuracy, depth, and clarity, averaging both turns per question.

D.3 Safety Benchmarks

The safety evaluation combines adversarial, jailbreak, and naturalistic harmful-request benchmarks with refusal-rate judging and bootstrap uncertainty estimates. The section also documents a verbosity bias in pointwise self-verification and replaces it with conservative pairwise comparison using position swapping.

Safety Evaluation: Safety alignment is evaluated with HarmBench, JailbreakBench, and PKU-SafeRLHF, covering adversarial, jailbreak, and naturalistic harmful requests.The evaluation uses 400 HarmBench prompts, 79 JailbreakBench prompts, and 400 PKU-SafeRLHF prompts.
Safety Evaluation: Refusal Rate (RR%) is the percentage of harmful requests that receive a refusal from the evaluated model.An LLM judge classifies each response as a refusal when it answers “Yes” to whether the harmful request was refused.
Safety Evaluation: Safety metrics use 1,000 bootstrap resamples to report mean ± standard error and 95% confidence intervals.This accounts for sampling variability in the judged evaluation sets.
Self-Verification: Pointwise self-judging systematically favored longer answers, even when they were factually incorrect, inflating persona distillation decisions.For Mistral-7B, the math persona had a 68% distill rate while external MT-Bench judging showed its math score falling from 9.05 to 6.10.
Self-Verification: PRISM replaces pointwise scoring with pairwise comparison and position swapping, accepting the expert only when it wins in both answer orders.The design enables direct comparison, reduces position bias, and filters disagreements between orderings.

F Gated Single-LoRA Training Setup

The Gated Single-LoRA setup combines one LoRA adapter with a binary per-query gate that decides whether to activate persona behavior. Training uses distill and retain samples with gate, distillation, and retention objectives, while precomputed teacher logits reduce memory demands.

Architecture: The Gated Single-LoRA variant uses one higher-rank LoRA adapter controlled by a binary gate instead of multiple expert adapters.
Architecture: The gate is a 3-layer MLP with GELU activations that operates on the last-token hidden state from transformer layer 0.The adapter covers attention and MLP projections, while the gate decides activation per query.
Training Data: Training partitions examples into distill samples where a persona beats the baseline and retain samples where the baseline is best.For Qwen2.5-7B-Instruct, the split contains 282 distill and 318 retain samples.
Training Objective: The objective combines binary cross-entropy gate loss with KL distillation and KL retention losses, using λretain = 0.5 for retain samples.Teacher logits are precomputed and stored to avoid out-of-memory failures during training.
Compute: The experiments run on single-GPU nodes, with lightweight gate training and stored teacher logits reducing memory requirements.The reported hardware includes NVIDIA A100 80GB and RTX A6000 48GB GPUs.