Source-linked AI summary
Enhancing SAE-based Steering via Neighbor Integrated Feature Selection
Yutian Liu, Xu Wang, Difan Zou
TL;DR
SAE-based steering relies on feature selection, but statistical top-k rankings may overlook effective features distributed among representationally adjacent groups. The paper proposes NIFS, which integrates similar neighboring features into steering, and reports consistent improvements across models and tasks while preserving generation quality. Its evaluation is limited to relatively small language models, open-ended tasks, and existing SAE and steering frameworks.
Problem
Feature selection is critical in SAE-based steering, yet the effectiveness of statistical top-k strategies has not been rigorously validated and may overlook influential features.
Method
NIFS selects top-ranked core features, retrieves representation-similar neighboring features, and integrates them into the steering vector with similarity-based weights.
Results
NIFS consistently improves steering performance across models and tasks while preserving generation quality.
Takeaways & Limitations
Representation structure, including adjacent groups created by feature splitting, should inform feature selection for controllable SAE-based steering.
Takeaways & Limitations
Experiments use a limited set of open-ended tasks and relatively small language models, leaving effectiveness on larger frontier models and broader scenarios unexplored.
Abstract
from arXiv · showhide
Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most existing SAE-based steering methods select features by applying a top- filter based on statistical scores, assuming that higher-scoring features yield stronger steering effects. In this paper, we show that this assumption is often invalid, leading to suboptimal feature selection. Our analysis reveals that effective steering features may be distributed among representationally adjacent, semantically similar groups induced by feature splitting in SAEs. Within such groups, features may exhibit disparate statistical scores despite having comparable steering influence, causing score-based selection to overlook important features. Based on these observations, we propose \textsc{Neighbor Integrated Feature Selection} (\textsc{NIFS}), a plug-and-play strategy that leverages representation similarity to improve feature selection for steering. We evaluate \textsc{NIFS} across multiple SAE-based steering methods and tasks, and demonstrate consistent performance gains over conventional top-$k$ selection.
1 Introduction
SAE-based steering depends critically on feature selection, yet statistical top-k rankings can miss effective features because steering-relevant representations may be distributed across similar, split features. NIFS addresses this by integrating representation-similar neighbors into steering.
- Motivation: SAE-based steering manipulates interpretable SAE features, making feature selection central to steering controllability and effectiveness.SAEs decompose LLM activations into interpretable dimensions, while steering methods manipulate activations associated with downstream tasks.
- Limitations of statistical selection: Statistical top-k selection is suboptimal: increasing k can saturate or degrade performance, while some sampled subsets outperform the original top-k set.Additional features may introduce negative steering effects that outweigh highly ranked features.
- Representational evidence: Representational similarity does not follow statistical rank perfectly, because several low-ranked features still align strongly with steering-relevant probing directions.Although similarity generally decreases with rank, lower-ranked features can retain high representational similarity.
- Feature splitting: Feature splitting can distribute a higher-level semantic concept across similar but functionally differentiated features with unequal statistical scores.Consequently, some split features may remain low-ranked despite substantial steering effectiveness.
- NIFS: NIFS selects statistically strong core features, retrieves representation-similar neighbors, and integrates them with similarity-based weights.The method is designed to recover fragmented steering signals while maintaining generation quality.
- Contribution: The paper proposes NIFS as a plug-and-play enhancement that integrates representation-similar features to improve steering across models and tasks.This contribution follows the finding that conventional statistical selection overlooks influential features.
2 Preliminaries
SAEs encode model representations as sparse, high-dimensional features that can support steering through selected contrastive feature activations. Steering modifies hidden representations by adding a vector, while SAE-based methods construct that vector from statistically selected features.
- Sparse Autoencoders: SAEs map hidden representations into a high-dimensional sparse feature space and reconstruct them through encoder and decoder components.Each decoder row can be interpreted as an SAE feature representing a concept.
- SAE-based Steering: Steering modifies a model hidden representation by adding a steering vector scaled by a steering-strength parameter.The vector is applied at inference time to influence model behavior.
- SAE-based Steering: CAA-style steering computes a vector from activation differences between positive and negative contrastive samples.The contrastive activation difference is transferred into the steering process.
- SAE-based Steering: SAE-based steering filters contrastive SAE activations using statistical feature-importance metrics before constructing the final steering vector.Selected positive and negative features are classified by their mean activation difference.
- Evaluation Context: Figure 1 reports Gemma2-2B steering success rates across top-k values from 100 to 1500 for Knowledge Conflicts, Sentiment, and Detoxification.Figure 2 reports success-rate distributions across 32 random feature-selection runs.
3 Experimental Setup
The experiments evaluate SAE-based steering on two language models across three open-ended tasks, using four statistical feature-selection baselines. The setup compares different criteria for identifying steering-relevant features.
- Models: Experiments use Gemma-2-2B and Llama-2-7B together with their corresponding SAEs.The models provide two base-model settings for evaluating steering methods.
- Tasks and Metrics: Evaluation covers Knowledge Conflicts, Sentiment, and Detoxification as three open-ended steering tasks.Dataset, evaluation-setting, and metric details are provided separately in the paper.
- Baselines: The baseline methods select features using statistical criteria based on activation differences, activated frequency, combined amplitude and frequency, or mutual information.The baselines are Diffmean, SAIF, STA, and SpARE.
4 Understanding Statistical Selection Strategies in SAE-based Steering
The analyses show that statistical top-k scores do not reliably identify the most effective steering features. Representational similarity helps explain this mismatch and motivates integrating neighboring features rather than treating ranked features independently.
- 4.2 Top-k selected features are suboptimal: Statistical top-k selection is suboptimal because steering performance often saturates or degrades as more features are added.Across k values from 100 to 1500, none of the evaluated methods improves consistently with increasing k.
- 4.2 Top-k selected features are suboptimal: Some randomized feature subsets outperform the original top-k selection, indicating that higher-effectiveness subsets can exist beyond the highest-ranked features.The comparison fixes k = 100 and repeats randomized selection 32 times.
- 4.2 Top-k selected features are suboptimal: Cosine similarity between SAE features and probing directions generally decays approximately exponentially with feature rank, while top-ranked features show the strongest alignment.Figure 3 displays individual features and an exponential fit for positive and negative probing directions.
- 4.2 Top-k selected features are suboptimal: Several low-ranked features still have high probing-direction similarity, showing that statistical ranking does not fully capture steering-relevant representations.These features can remain low-ranked despite substantial steering effectiveness.
- 4.2 Top-k selected features are suboptimal: Feature splitting can distribute semantically related steering features across different statistical ranks because some split features receive stronger activation responses than others.This provides a structural explanation for why score-based selection can overlook useful neighboring features.
- Neighbor Integration: NIFS begins with statistically ranked core features, retrieves their most similar neighbors from a candidate pool, and forms an enhanced feature set.The algorithm uses a pool size α and neighbor size β to perform nearest-neighbor integration.
5 NIFS: Neighbor Integrated Feature Selection
NIFS augments score-based SAE feature selection by integrating representationally similar neighbors and reweighting their steering contributions. It is designed to recover low-scoring but functionally relevant features while preserving the stability of statistical ranking.
- Neighbor Integration: NIFS expands conventional statistical selection with semantically similar neighboring SAE features identified through representation similarity.The method uses a pool anchored to score-based ranking, then retrieves nearest neighbors using cosine similarity.
- Method Overview: NIFS is presented as a plug-and-play enhancement for SAE-based steering methods that rely on statistical feature selection.Its stated purpose is to integrate representation structure without sacrificing the stability of score-based selection.
- Motivation: The design treats statistical scores as informative coarse signals while addressing feature splitting, which can leave effective features individually low-ranked.Feature splitting can distribute related steering functionality across semantically similar components with different statistical scores.
- Neighbor Integration: The method divides selected features into a statistically supported core set and a representation-based neighbor set.Core features are traversed in descending rank, while neighbors are retrieved from the candidate pool until the selection budget is met.
- Feature Reweighting: NIFS reweights neighbor steering coefficients according to their cosine similarity with core features, reducing the influence of weakly aligned neighbors.This balances broader feature coverage against noise from neighbors with lower input sensitivity.
6 Experiments
Experiments evaluate NIFS across SAE-based baselines, models, tasks, neighbor sizes, and ablations. NIFS generally improves steering, preserves fluency, and benefits from similarity-based neighbor integration without favoring simple feature-count expansion.
- Overall Performance Comparison: NIFS consistently improves steering performance across evaluated SAE-based baselines, models, and tasks.On Llama2-7B, STA gains 2.15% SR on Sentiment and 2.21% SR on Detoxification; improvements also appear on Gemma2-2B and Gemma3-12B.
- Overall Performance Comparison: NIFS preserves generation quality while improving steering effectiveness, with fluency variations remaining within 0.1 across sentiment control and detoxification.The paper evaluates n-gram fluency alongside steering success rate.
- Overall Performance Comparison: 80.80 to 85.30 (+4.50) is the reported SR improvement for SPARE on Llama2-7B detoxification after applying NIFS.The paper notes that gains are especially pronounced for strong baselines and challenging tasks.
- Effects of Neighbor Size: Average core-neighbor similarity exceeds the Johnson–Lindenstrauss threshold ϵ = 0.24 across a wide range of neighbor sizes.This suggests that NIFS neighborhoods reflect meaningful feature structure rather than random high-dimensional geometry.
- Effects of Neighbor Size: Steering performance does not change monotonically with neighbor size: moderate integration helps, whereas overly large neighborhoods dilute the steering signal.Larger neighborhoods can introduce features with weaker steering relevance.
- Ablations: NIFS outperforms Core-Only, Core-Random, and Larger-K, indicating that similarity-based neighbor selection is more effective than random or simple feature-count expansion.The Larger-K comparison also associates lower-ranked features with weak or negative steering effects.
- Ablations: Removing Feature Reweighting generally degrades steering performance, although the ablated variant still exceeds corresponding baseline methods.The comparison isolates the reweighting module while keeping other NIFS components unchanged.
7 conclusion
The paper finds statistical top-k selection suboptimal because effective steering features can form representationally adjacent groups through feature splitting. NIFS integrates similar features and improves steering performance while preserving generation quality.
- Feature splitting creates representationally adjacent groups whose influential components may be missed by purely score-based selection.
- NIFS augments statistical selection with representation-aware feature integration as a plug-and-play steering strategy.
- NIFS improves steering performance while preserving generation quality across the evaluated settings.
8 Limitations
The experiments cover a limited set of open-ended steering tasks and relatively small-scale language models. The analysis relies primarily on existing SAE architectures and steering methods, leaving generalization to other frameworks for future work.
- The experiments are limited to a small set of open-ended steering tasks and relatively small-scale language models.
- The analysis is primarily based on existing SAE architectures and steering methods.
- Whether the observed phenomena generalize to other representation learning and steering frameworks remains for future investigation.
B.2 Task, Dataset, and Evaluation Details
The evaluation uses three open-ended steering tasks—Knowledge Conflicts, Sentiment, and Detoxification—with task-specific datasets and success-rate definitions. Fluency is additionally measured for Sentiment and Detoxification using an n-gram metric.
- Tasks and Datasets: The evaluation covers Knowledge Conflicts, Sentiment, and Detoxification as three open-ended steering tasks.
- Tasks and Datasets: Knowledge Conflicts uses NQSwap to test whether models resolve discrepancies between contextual and parametric knowledge.
- Tasks and Datasets: Sentiment uses labeled movie reviews and a 961-review evaluation set, while Detoxification uses Paradetox for steering-vector construction and RealToxicPrompts for evaluation.
- Evaluation and Metrics: Success rate is the primary metric across tasks, measuring contextual-knowledge use for Knowledge Conflicts and positivity scores for Sentiment.
- Evaluation and Metrics: Detoxification success rate is the non-toxicity rate, and an n-gram metric evaluates fluency for Sentiment and Detoxification.
B.4 Hyperparameters in Main Experiments
The main experiments choose feature budgets and neighbor sizes using performance-based criteria, with NIFS using a fixed pool expansion ratio. Figures examine steering success across top-k values, random-selection runs, and feature-rank similarity.
- Hyperparameters: Feature budgets use task- and method-specific optimal k values, except DiffMean on Llama2-7B Detoxification, where k = 100 avoids degraded generation quality.
- Hyperparameters: NIFS uses pool expansion ratio α = 2, while neighbor size β is selected by validation performance.
- Result Interpretation: The unusually high Diffmean performance on Llama2-7B Detoxification is attributed to model collapse and failure to generate meaningful outputs.
- Visualizations: Additional visualizations compare feature rank with probing-direction similarity across models and tasks, with most methods following a similar trend except Diffmean on several tasks.
- Visualizations: Figure 5 shows Llama2-7B steering success rates across top-k values from 100 to 1500.
- Visualizations: Figure 6 shows the distribution of success rates over 32 random feature-selection runs on Llama2-7B.
C.4 Ablation Results of Feature Reweighting module
The ablation study evaluates feature reweighting across tasks on Gemma2-2B. NIFS with feature reweighting outperforms the variant without it in most comparisons.
- 10/12 comparisons favor NIFS with feature reweighting over the variant without reweighting on Gemma2-2B.The study reports this as a consistent overall benefit despite a few exceptions.
- Table 7 reports the feature-reweighting ablation across different tasks on Gemma2-2B.