PaperlayerView exact revision on arXiv

Source-linked AI summary

GEOSTEER: Geodesic Optimization for Activation Steering in Large Language Models

Xuan Cuong Ngo, Hao Vo, Ngan Le

arXiv:2609.10658v1cs.LG

TL;DR

Existing norm-preserving steering methods are constrained by predefined trajectories and one-step updates that may miss complex activation structure. GeoSteer uses adaptive multistep Riemannian optimization with geodesic updates, and it consistently improves over strong baselines across three benchmarks while preserving activation norms.

  • Problem

    Existing norm-preserving steering methods rely on predefined trajectories and one-step updates that may fail to capture complex activation distributions.

  • Method

    GeoSteer formulates steering as Riemannian optimization and uses a learned nonlinear objective to guide adaptive geodesic updates on the activation sphere.

  • Results

    GeoSteer consistently improves over strong activation steering baselines across TruthfulQA, RealToxicityPrompts, and UltraFeedback while preserving activation norms.

  • Takeaways & Limitations

    Replacing predefined one-step edits with adaptive, geometry-aware optimization provides a smoother and more stable approach to norm-preserving activation steering.

  • Takeaways & Limitations

    Activation steering can be misused to induce harmful behaviors, and preserving activation norms does not guarantee avoidance of unintended behavioral changes.

Abstract

from arXiv · show

Activation steering provides a lightweight way to control large language models (LLMs) by modifying their hidden activations at inference time. Among these approaches, norm-preserving steering aims to change model behavior without altering the activation norm, reducing the risk of representation collapse and degradation. However, existing norm-preserving methods are limited by predefined steering trajectories and by their reliance on one-step updates, which may fail to capture the complex structure of activation distributions. We propose GeoSteer, an optimization-based method for norm-preserving activation steering. GeoSteer formulates steering as a Riemannian optimization problem and updates activations through a sequence of small geodesic steps on the representation manifold. To avoid fixed steering directions, GeoSteer learns a nonlinear activation-space objective that distinguishes desired from undesired activations, and uses this function to adaptively guide each steering step. This multistep formulation yields smoother, more stable, and more consistent steering behavior while preserving the activation norm. Across TruthfulQA, RealToxicityPrompts, and UltraFeedback benchmarks, GeoSteer consistently improves over state-of-the-art activation steering baselines. These results suggest that norm-preserving steering can be made more effective by replacing predefined one-step edits with adaptive, geometry-aware optimization.

1 Introduction

GEOSTEER addresses limitations of norm-preserving activation steering by replacing predefined one-step trajectories with adaptive, multistep geodesic optimization. It preserves activation norms and consistently improves over strong steering baselines across three benchmarks.

  • Activation steering offers lightweight inference-time behavioral control without updating model parameters, motivating norm-preserving alternatives to fine-tuning.
  • Norm-preserving steering avoids unconstrained shifts but existing methods often use heuristic trajectories, one-step updates, or restrictive geometric assumptions.These limitations can ignore richer activation-distribution structure and reduce flexibility across models or tasks.
  • A nonlinear activation-space objective distinguishes desired from undesired activations and guides iterative steering toward desired behavior.The objective enables finer-grained optimization of activation-space structure than predefined rotations.
  • GEOSTEER formulates norm-preserving steering as Riemannian optimization on the activation sphere, providing a geometric view of activation intervention.
  • GEOSTEER replaces fixed transformations with small geodesic updates whose directions adapt to the current activation state.The updates form a smooth trajectory while preserving the activation norm.
  • GEOSTEER consistently improves over strong activation steering baselines on TruthfulQA, UltraFeedback, and RealToxicityPrompts while maintaining efficient inference.

2 Related Work

Activation steering commonly modifies hidden states with contrastive direction vectors during decoding. GEOSTEER instead recomputes the steering direction through adaptive multistep updates on the activation manifold.

  • Most activation-steering methods estimate a direction from contrastive activation patterns and add a scaled vector to hidden states during decoding.
  • GEOSTEER recomputes the steering direction at each step from the current activation while updating on the manifold.

3 Methodology

GEOSTEER frames activation steering as constrained optimization on a hypersphere, combining a learned nonlinear objective with iterative geodesic updates. This preserves activation norms while adapting the steering direction to the current activation.

  • 3.1 Preliminaries: Conventional additive steering changes hidden activations by adding a scaled direction, whereas norm-preserving steering constrains edited representations to the original-norm hypersphere.The additive update may alter both activation direction and norm; spherical steering instead modifies the normalized direction.
  • 3.1 Preliminaries: GEOSTEER treats steering as Riemannian optimization over normalized activation directions, then reconstructs the edited activation with its original norm.The method operates on the unit sphere and restores the original activation norm after geodesic updates.
  • 3.2 GEOSTEER: A nonlinear probe distinguishes desired from undesired activations and supplies the objective used to guide inference-time steering.The probe uses a nonlinear feature map with learnable parameters and is trained with binary classification loss.
  • 3.2 GEOSTEER: At each inference step, GEOSTEER projects the gradient onto the sphere’s tangent space and moves along a geodesic, keeping intermediate activations on the norm-preserving manifold.The procedure begins from z(0), uses K steps with η = T/K, and applies geodesic updates after tangent-space projection.
  • 3.3 Comparison with Existing Norm-Preserving Steering: The resulting trajectory adapts to each activation through local gradients rather than using a single global direction or fixed two-dimensional rotation plane.This replaces fixed geometric transformations with a curved, activation-dependent path toward the desired region.

4 Experiments

GEOSTEER is evaluated on helpfulness, truthfulness, and detoxification across multiple open-source models, with ablations examining its design choices and runtime analysis measuring inference overhead.

  • Experimental Results: GEOSTEER achieves the best primary performance across evaluated models, with the highest UltraFeedback win rate, TruthfulQA T×I, and lowest RealToxicityPrompts toxicity.The comparison covers helpfulness, truthfulness, and detoxification across Falcon-7B, Mistral-7B, and LLaMA3.1-8B.
  • Experimental Results: GEOSTEER improves over additive and linear steering while preserving activation norms through hypersphere-based updates and adaptive geodesic trajectories.The reported comparison includes CAA, MiMiC, ITI, Linear-AcT, HPR, Angular Steering, and Spherical Steering.
  • Ablation Studies: Increasing geodesic steps provides a modest initial gain before performance stabilizes, indicating that a moderate number of steps is sufficient for reliable integration.Figure 2 evaluates geodesic-step count and total steering strength on TruthfulQA using True × Info.
  • Ablation Studies: Polynomial Count Sketch performs best among the tested objectives, reaching 73.3 True and 69.4 True × Info while keeping Info nearly unchanged.The ablation compares a linear objective with Random Fourier Features and Polynomial Count Sketch on LLaMA3.1-8B.
  • Runtime analysis: GEOSTEER retains nearly all original generation throughput, averaging 100.7 tokens/s versus 101.6 tokens/s for unmodified models, with 0.9% average overhead.Runtime is measured during autoregressive generation across Falcon-7B, Mistral-7B, and LLaMA3.1-8B on four NVIDIA A100 40GB GPUs.

5 Conclusion

GEOSTEER formulates norm-preserving activation steering as adaptive Riemannian optimization using small geodesic updates guided by a learned nonlinear objective. Across three benchmarks, it consistently improves over strong baselines while maintaining modest inference cost.

  • 5 Conclusion: GEOSTEER adaptively recomputes steering directions from a learned nonlinear objective while moving activations through small geodesic steps on the activation sphere.This replaces predefined one-step edits with multistep optimization that preserves activation norms.
  • 5 Conclusion: GEOSTEER consistently improves over strong activation steering baselines across TruthfulQA, RealToxicityPrompts, and UltraFeedback.The conclusion also reports that ablations support the importance of geodesic steps, steering strength, and nonlinear objective design.

6 Limitations

GEOSTEER has limitations in integrating alternative feature-discovery methods and in fully characterizing safety and unintended-behavior risks. The authors identify SAE integration as technically challenging and call for broader safety evaluation before deployment.

  • 6 Limitations: Integrating GEOSTEER with sparse-autoencoder features is non-trivial because SAE representations are high-dimensional, sparse, and featurewise rather than direction-wise.The authors note that stable feature combinations must be selected and converted into smooth tangent directions without introducing noise or instability.
  • 6 Limitations: The method may be misused to induce toxic, biased, deceptive, or otherwise harmful outputs, and norm preservation does not guarantee that unintended behavioral changes are avoided.The authors therefore call for downstream safety, robustness, and misuse evaluations before deployment.
  • 6 Limitations: Future work will investigate SAE-based feature discovery and broader safety evaluations across tasks and model families.These evaluations are intended to better characterize unintended behavioral changes and misuse risks.

A.1 LLM Usage

LLMs were used only for manuscript polishing, readability improvements, and small portions of experimental code. The authors retained responsibility for the research ideas, methods, experiments, analysis, and conclusions.

  • A.1 LLM Usage: LLMs assisted with grammar, clarity, readability, and small portions of experimental code, but were not used to generate the core scientific contributions or conclusions.The authors state that all LLM-assisted content was reviewed, verified, and revised.
  • A.1 LLM Usage: The authors developed the research ideas, theoretical framework, methodology, experimental design, and analysis.They also take full responsibility for the correctness of the theoretical claims and reported results.
  • A.1 LLM Usage: LLM-produced text or code was checked for correctness, originality, and compliance with ethical research standards.

A.2 Scientific Artifacts

The study uses publicly available datasets, pretrained models, evaluation models, and baseline implementations under their respective licenses. It avoids new human-subject data collection and reports aggregate results rather than individual examples.

  • A.2 Scientific Artifacts: The evaluation uses TruthfulQA for truthfulness, RealToxicityPrompts for toxicity-oriented generation, and UltraFeedback for preference-oriented evaluation.
  • A.2 Scientific Artifacts: The experiments use publicly available models and artifacts under their respective licenses, including Apache-2.0, MIT, and the LLaMA 3.1 Community License.The models include Falcon-7B, Mistral-7B, Qwen2.5-7B, and LLaMA3.1-8B.
  • A.2 Scientific Artifacts: The study collects no new human-subject data and reports aggregate evaluation results rather than individual examples.Some benchmark data, especially RealToxicityPrompts, may contain sensitive or offensive text by construction.

A.3 PCS Hyperparameters

The study uses Polynomial Count Sketch to approximate polynomial features efficiently without explicitly expanding high-dimensional features. The same configuration is used across datasets and base models.

  • A.3 PCS Hyperparameters: Polynomial Count Sketch provides an efficient approximation to the polynomial kernel without explicitly expanding high-dimensional polynomial features.
  • A.3 PCS Hyperparameters: The experiments set γ = 0.1, c0 = 1.0, d = 2, and Npoly = 8000 for the randomized polynomial features.Here, γ controls inner-product scale, c0 is the constant offset, d is the polynomial degree, and Npoly is the number of random features.
  • A.3 PCS Hyperparameters: The selected Polynomial Count Sketch configuration was stable across different datasets and base models.

A.4 Norm Preservation of GeoSteer

GeoSteer preserves activation norms by iteratively updating normalized activation directions on the unit sphere. The section also specifies shared steering-layer selection and evaluation settings across models and benchmarks.

  • Norm Preservation: The method obtains each update direction by projecting the gradient onto the sphere’s tangent space and normalizing it.This keeps optimization constrained to the spherical representation manifold.
  • Norm Preservation: GeoSteer preserves the original activation norm because every intermediate direction remains on the unit sphere before final reconstruction.The proof starts from a normalized activation, uses tangent-space updates, and establishes sphere membership at every step.
  • Steering Position: CAA selects shared steering layers—15 for Falcon-7B, 16 for Mistral-7B, and 14 for both LLaMA3.1-8B and Qwen2.5-7B—before method comparisons.All methods are evaluated at the same residual-stream position within each model rather than tuning layers separately.
  • Evaluation Setup: Experiments evaluate TruthfulQA and UltraFeedback with fixed generation settings and reward-based or truthfulness-informativeness metrics.UltraFeedback reports RM-mean, RMP90, and reward-model win rate, while TruthfulQA uses truthfulness and informativeness judges.
Loading 2609.10658v1…