Source-linked AI summary
The Invisible Editorial Layer: Formalizing Undisclosed Inference-Time Steering, Probability Placement, and the Attribution Problem in Deployed Language Models
Augusto Camargo
TL;DR
The paper addresses the gap between auditing a foundation model and auditing the composite deployed system that produces observed outputs. It formalizes runtime steering and black-box attribution limits, then characterizes Probability Placement as undisclosed commercial influence embedded in an ostensibly organic assistant response. Its scope is conceptual: it establishes architectural feasibility and attribution limits, not evidence of current provider misconduct.
Problem
Behavior observed through commercial interfaces does not necessarily characterize the underlying model, while black-box samples leave the responsible deployment components latent.
Method
The paper develops a conceptual and formal framework for runtime inference steering, composite deployed systems, observational non-identifiability, Probability Placement, and inference-policy verification.
Results
Black-box behavioral shifts do not uniquely identify their architectural source, and Probability Placement describes undisclosed commercial influence embedded within an ostensibly organic assistant response.
Takeaways & Limitations
AI auditing and governance should treat the served inference pipeline, rather than only model weights, as the relevant object of analysis.
Takeaways & Limitations
The paper is conceptual and demonstrates architectural feasibility, not evidence that major providers currently deploy undisclosed political or commercial logit steering.
Abstract
from arXiv · showhide
Evaluations of generative language models frequently interpret observable behavioral traits, such as political stance, brand inclination, and normative framing, as manifestations of model weights, post-training alignment, or prompting. This interpretation risks conflating a foundation model with the multi-layered production system through which its outputs are ultimately served. Modern inference stacks support runtime interventions capable of modifying generation while model parameters remain frozen. We examine inference-time framing bias: systematic runtime steering of generated text toward institutional, ideological, or commercial frames without requiring changes to the underlying model parameters. We formalize the Inference Attribution Problem and establish an observational non-identifiability result showing that, under black-box observation alone, behaviorally equivalent deployed systems may arise from structurally distinct combinations of model parameters and inference policies. Consequently, observed behavioral bias does not uniquely identify the architectural layer responsible for it. We further characterize Probability Placement as a deployment pattern in which undisclosed commercial influence is embedded within an ostensibly organic assistant response through systematic probability-mass reallocation, distinguishing it from explicit token-auction mechanisms for generative advertising. Finally, we discuss implications for behavioral auditing, inference provenance, confidential computing, cryptographic attestation, the EU AI Act, the Digital Services Act, and advertising-disclosure principles. We argue that governance of generative systems must increasingly distinguish between auditing a model and auditing the deployed system that ultimately speaks.
1 Introduction
Deployed language systems combine foundation models with runtime inference layers, so observed behavior may not originate in model parameters alone. The paper names this attribution ambiguity and frames auditing around the full deployed system.
- 1 Introduction: Model behavior is not necessarily deployed-system behavior because production interfaces may add runtime transformations to a foundation model.These transformations can modify generation before token selection or during forward computation without changing parameters.
- 1 Introduction: Inference-time steering can alter generation while model parameters remain frozen and may be difficult to distinguish from weight- or alignment-originated behavior.The paper connects this possibility to controllability, safety, personalization, watermarking, and provenance mechanisms.
- 1 Introduction: The Inference Attribution Problem is the structural ambiguity of determining which architectural layer produced an observed ideological, commercial, or institutional preference.Black-box behavioral evidence alone cannot distinguish among training, prompting, retrieval, activation, logit, and sampling sources.
- 1 Introduction: The paper contributes a framework for runtime steering, observational attribution limits, Probability Placement, detection–attribution separation, and governance criteria for deployed inference pipelines.Its governance discussion includes inference transparency, cryptographic attestation, confidential computing, and related regulatory implications.
2 Related Work
Prior work shows that frozen language models can be steered through decoding, representations, logits, and watermarking. The paper distinguishes its Probability Placement concept from disclosed token-auction advertising by focusing on undisclosed influence blended into an assistant response.
- 2 Related Work: Controlled-generation methods steer desired attributes through gradients, discriminators, expert models, predictors, representation interventions, or direct logit modification.These approaches operate during inference rather than requiring permanent parameter changes.
- 2 Related Work: Frozen-model behavior can be materially altered through inference-only mechanisms, establishing runtime steering as an architectural possibility.The cited methods include both token-distribution and internal-representation interventions.
- 2 Related Work: Watermarking provides a production precedent for sustained token-distribution perturbation while preserving overall generation quality.Watermark systems bias sampling toward pseudorandomly preferred vocabulary subsets without representing ideological or commercial steering.
- 2.3 Generative Advertising and Token Auctions: Probability Placement differs from token auctions because sponsored influence is undisclosed and embedded within an otherwise general-purpose assistant’s ostensibly organic response.Token auctions explicitly model advertisers as participants generating advertising content.
- 2 Related Work: Framing research and conversational persuasion studies motivate examining selective salience and framing choices alongside factual accuracy.The cited work links conversational language systems to changes in user beliefs and attitudes.
3 Mechanisms of Inference-Time Framing
The paper models inference-time framing as a transformation of the base token distribution before sampling. Unlike hard suppression, probabilistic steering can reweight competing frames while leaving alternatives possible.
- 3.1 Formalizing Logit-Level Interventions: A language model defines a conditional token distribution from context, prior tokens, and raw logits, which an inference policy may transform before sampling.The policy can use external scores, user or interaction information, steering objectives, and an intervention magnitude.
- 3.1 Formalizing Logit-Level Interventions: The served distribution is obtained by reweighting the base distribution with exp(λs_t), so steering changes probability mass without requiring parameter updates.The scoring function may derive from watermarking rules, classifiers, latent representations, vocabulary projections, or related objectives.
- 3.2 Probabilistic Salience vs. Hard Suppression: Hard suppression makes prohibited outputs impossible and can therefore create identifiable boundaries.This contrasts with semantic steering that need not eliminate alternative outputs.
- 3.2 Probabilistic Salience vs. Hard Suppression: Semantic steering can favor one framing over another without making the alternative framing impossible.The paper illustrates safeguard and burden frames for regulation-related policy queries.
- 3.2 Probabilistic Salience vs. Hard Suppression: An inference policy can influence interpretation by making selected facts, descriptors, examples, and causal relationships more probable rather than fabricating information.The underlying parameters may remain unchanged while targeted topics receive altered interpretive salience.
4 Deployment Paradigms and Threat Models
The paper describes deployment patterns in which runtime inference policies alter targeted framing or commercial recommendations while model weights remain unchanged. These interventions can personalize persuasion and embed commercial influence within an assistant’s ostensibly organic language.
- 4.1 State-Enforced Framing Mandates: Runtime policies can alter interpretive salience on targeted topics without changing underlying parameters.The deployed system may remain behaviorally ordinary on unrelated prompts.
- 4.2 Personalized Persuasion: Intervention strength can depend on inferred user receptivity, enabling individualized persuasive behavior without separate weights for each group.The architecture uses lower or higher intervention strength for users estimated to resist or accept the target frame.
- 4.3 Commercial Framing: Probability Placement: Probability Placement embeds commercial influence into the probability distribution of a general-purpose conversational assistant.The paper presents the distributions as illustrative rather than empirical.
- 4.3 Commercial Framing: Probability Placement: Unlike explicit token auctions, Probability Placement lets an undisclosed commercial policy influence an assistant that appears to provide organic recommendations or judgments.Competing entities can remain possible even as probability mass shifts toward a commercially preferred entity.
- 4.3 Commercial Framing: Probability Placement: Commercial influence can affect entity selection, favorable attribute associations, and comparative salience.These dimensions include ranking preferred entities first, pairing them with favorable descriptors, and surfacing competitor disadvantages selectively.
- 4.3 Commercial Framing: Probability Placement: Embedded interventions need not produce a visually separable advertising unit because their commercial effect can appear as the assistant’s linguistic judgment.This distinguishes the deployment pattern from conventional sponsored results.
5 The Inference Attribution Problem
The paper formalizes the Inference Attribution Problem: deployed behavior reflects a composite inference system, while black-box observations reveal only its served distribution. Structurally different combinations of model parameters and runtime policies can therefore produce behaviorally equivalent outputs.
- Composite deployed systems: A production endpoint combines model parameters, post-training effects, hidden instructions, retrieval, activation interventions, logit policies, and sampling configuration.The paper represents the deployed distribution as a function of these components.
- The attribution problem: A behavioral auditor generally observes samples from the served distribution, while the decomposition responsible for those samples remains latent.
- Observational non-identifiability: For any supported target distribution, a non-trivial inference policy can make a base model expose that distribution.The proposition establishes the existence of a logit-level policy for the target distribution.
- Observational non-identifiability: The same served distribution is compatible with either a steered base model or a different model implementing the target distribution with identity inference.
- Auditing implications: Black-box observations alone cannot uniquely identify whether observed behavior originates from model parameters or runtime steering.Additional assumptions, privileged access, reference execution, instrumentation, or attestation are needed to distinguish structurally different implementations.
- Auditing implications: Logit-level steering can alter generation without adding prompt tokens visible or extractable by an ordinary user, although other audit channels may expose it.
6 Auditing, Detection, and Attribution
Auditing can establish behavioral divergence in a served system, but not necessarily the architectural locus of that divergence. The paper discusses distributional comparisons and semantic repeated-generation methods for detecting systematic shifts under different observability conditions.
- Detection is not attribution: Behavioral divergence does not establish whether the cause is a model checkpoint, post-training, system prompt, retrieval, activation intervention, or sampling configuration.
- Detection is not attribution: Black-box ideological-bias audits can measure systematic behavioral asymmetries without determining where they originate in the serving stack.
- Distributional divergence metrics: When token probabilities are accessible, auditors can quantify divergence between production and reference distributions using Kullback–Leibler divergence and Total Variation distance.
- Distributional divergence metrics: When endpoints expose only sampled text, auditors can estimate semantic distributional shifts across repeated generations.The framework compares competing frames using validated sentence-level representations.
- Distributional divergence metrics: Repeated measurements across paraphrases, languages, geographic origins, account states, and randomized interaction histories can help identify systematic asymmetries.
- Detection is not attribution: Even statistically convincing asymmetry remains evidence about the served system rather than necessarily its architectural provenance.
7 Verifiable Inference and Runtime Transparency
Because black-box observation cannot generally resolve implementation-level attribution, the paper proposes stronger observability through verifiable information about the inference environment. Attestation can reduce uncertainty about execution identity, but it cannot determine whether a policy is acceptable.
- Runtime transparency: Stronger governance models may require additional observability beyond black-box behavioral observation.
- Runtime transparency: An Inference Policy Transparency framework could attest the execution environment, model identity, active inference policies, and policy changes.The proposed components include measured execution environments, cryptographic commitments, and tamper-evident change logs.
- Runtime transparency: An attestation can bind commitments to the model, inference policy, sampler configuration, serving code, and execution time.
- Limits of attestation: A trusted execution environment may establish that a specific runtime policy executed, but not that it is politically neutral, commercially fair, scientifically justified, or legally permissible.
- Limits of attestation: Cryptographic verification complements institutional oversight by reducing uncertainty about whether the declared inference stack executed.
- Limits of attestation: Human, legal, or regulatory evaluation remains necessary to determine whether the declared policy is acceptable.
8 Regulatory Implications
Inference-time steering creates regulatory questions because probabilistic framing can shape recommendations without appearing as a distinct advertising object or obvious manipulation. Existing legal frameworks may not map cleanly onto these mechanisms, so disclosure obligations remain context-dependent.
- EU AI Act: Article 5(1)(a) of the EU AI Act prohibits certain subliminal, purposeful manipulative, or deceptive AI practices when statutory conditions are satisfied.Applicability depends on behavioral distortion, harm, purpose, affected population, and legally relevant consequences.
- EU AI Act: Probabilistic framing may be difficult to map onto legal frameworks designed around visible manipulation or discrete decisions.A subtle probability shift may lack individually obvious or immediately measurable injury, while repeated effects can aggregate at scale.
- Digital Services Act: Conversational assistants collapse retrieval, ranking, synthesis, framing, and recommendation into a single generated response.This complicates transparency models developed for recommender systems, where ranking and information selection shape what users see.
- Digital Services Act: Future transparency regimes could disclose material runtime policies that systematically affect which entities, arguments, or frames are favored during generation.The proposal extends transparency attention beyond retrieval or ranking parameters to the serving policies shaping generated outputs.
- Commercial Disclosure and Advertising Principles: When commercial consideration materially influences recommendation distribution, users should distinguish sponsored influence from otherwise organic generation.Probability Placement can embed commercial influence within the distribution constructing the assistant’s narrative, collapsing editorial and sponsored content.
9 Discussion
The Inference Attribution Problem changes AI auditing from examining model behavior alone to examining the entire serving stack. Runtime policies and deployment changes can produce divergent behavior across contexts, while black-box evidence cannot generally locate the causal layer.
- From Model Audits to System Audits: The Inference Attribution Problem changes the unit of analysis from model auditing to auditing the deployed system that ultimately speaks.The two questions overlap but are not equivalent.
- From Model Audits to System Audits: A model checkpoint can behave differently across providers, regions, cohorts, account states, product tiers, or time periods when surrounding inference policies differ.Conversely, distinct checkpoints can produce behaviorally similar outputs through runtime interventions.
- Temporal Deployment Variation: Behavioral audits need reproducibility information about model identity, deployment configuration, and time.A claim that “Model X exhibited bias B” may be underspecified when behavior came from a mutable serving environment.
- Scope and Limitations: The paper is conceptual and does not establish that major providers currently deploy undisclosed political or commercial logit-steering mechanisms.Its threat models demonstrate architectural feasibility rather than actual misconduct.
- Scope and Limitations: Black-box non-identifiability is a general limit, but log probabilities, open weights, reproducible checkpoints, policy documentation, or auditable code can reduce the attribution problem.Future work should investigate protocols for distinguishing runtime-intervention classes under partial observability.
10 Future Research
The paper proposes empirical protocols for detecting runtime divergence, commercial preference, and framing asymmetries, alongside provenance metadata that can make deployment changes auditable without necessarily revealing proprietary policies.
- Empirical Detection: Controlled prompts comparing a reference model with a production endpoint could reveal systematic divergence across topics, entities, demographics, regions, or account features.The proposed tests target where divergence concentrates under partial observability.
- Empirical Detection: Symmetry tests can compare semantically mirrored prompts for competing brands and test whether preferences persist after controlling for prompt order and factual attributes.Repeated sampling estimates recommendation probabilities such as P(A recommended | xA, xB).
- Framing Benchmarks: Future benchmarks could measure whether deployment systems reproducibly privilege one framing axis over alternatives without declaring any frame neutral.Suggested axes include innovation versus risk, regulation versus burden, and security versus liberty.
- Provenance Metadata: Standardized provenance metadata could complement model cards by documenting deployment configuration and changes between evaluation and production.Verifiable commitments could support authorized auditing without publicly exposing proprietary policy contents.
11 Conclusion
The paper argues that inference-time interventions decouple production behavior from foundation-model parameters, creating an attribution problem for black-box audits. It therefore treats the deployed inference pipeline—not only the model—as the relevant object for transparency and auditability.
- Conclusion: Modern serving layers can systematically modify observed text during inference without changing underlying model weights.Controlled generation, activation steering, decoding interventions, and watermarking establish this architectural possibility.
- Conclusion: Black-box behavior does not uniquely identify the responsible architectural layer because structurally distinct systems can be observationally equivalent at their outputs.A behavioral shift may be detected without identifying its causal locus.
- Conclusion: Probability Placement denotes undisclosed probability-level commercial influence embedded within an ostensibly organic assistant response.It differs from token-auction mechanisms in which advertisers explicitly participate in generative advertising markets.
- Conclusion: Governance frameworks should increasingly treat the deployed inference pipeline, rather than only foundation-model weights, as requiring transparency and auditability.The paper summarizes this distinction as: auditing the model is not auditing the system that speaks.