Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs

• Providers may covertly replace advertised models with cheaper alternatives, including smaller or heavily quantized variants, to reduce costs or increase margins.Undisclosed substitutions can also arise operationally, such as routing traffic from an overloaded model to a lower-tier one.Source: §1 · p. 1
• Black-box APIs expose prompts and text outputs, sometimes log probabilities, but not the model weights or underlying infrastructure.This information asymmetry makes it difficult for users or auditors to verify the served model.Source: §1 · pp. 1–2
• The paper formalizes black-box substitution detection and analyzes adversarial scenarios including quantization, randomized substitution, and benchmark evasion.Source: §1 · p. 2
• Text-based statistical tests lack power against subtle or randomized substitutions, while production inference nondeterminism defeats log-probability comparisons.The paper also proposes TEEs as a practical solution that provides provable model integrity with modest performance cost.Source: §1 · p. 2

• API monitoring and auditing spans capability tracking, output-distribution tests, identity-style prompts, self-queries, and activation-based proofs, but robustness under adversaries and production nondeterminism remains open.The cited approaches include Chen et al. (2023), Eyuboglu et al. (2024), Gao et al. (2024), Huang et al. (2025), Sam et al. (2025), and Ong et al. (2025).Source: §2 · p. 2
• Text detectors can struggle with subtle substitutions within one model family, while watermarking traces content ownership rather than enabling end-user model-identity verification.Source: §2 · p. 2
• Zero-knowledge proofs can verify inference without revealing inputs or weights but are described as too costly for real-time large-LLM APIs; TEEs offer lower-overhead hardware guarantees.Source: §2 · p. 2

3.1 PROBLEM FORMULATION

• The service model has a user or auditor, a black-box provider claiming to serve a target model, and the advertised model’s output distribution.Source: §3.1 · p. 3
• Substitution means undisclosed divergence between the actual backend distribution and the advertised model’s distribution; variants include smaller, quantized, fine-tuned, updated, or different-family models.Source: §3.1 · p. 3
• For proprietary APIs, auditors lack local access to the reference model, unlike open-source settings where they can generate reference outputs and log probabilities.Source: §3.1 · p. 3
• Auditing compares provider samples, and potentially reference samples, to determine whether the provider faithfully uses the specified model.Source: §3.1 · p. 3
• An effective verifier should combine low false-positive and false-negative rates, reasonable query budgets, resistance to countermeasures, and applicability across models and API configurations.Source: §3.1 · p. 3

3.2 ADVERSARIAL ATTACK SCENARIOS

• Quantization reduces memory footprint and can accelerate inference, while slightly changing outputs in ways that may evade naive detection with limited samples.Source: §3.2 · p. 3
• Randomized substitution mixes substitute and target outputs; as the substitution probability falls, the observed distribution approaches the target and sampling-based detection becomes harder.Source: §3.2 · p. 3
• Providers can evade fixed-prompt audits by identifying benchmark or identity queries and routing them to the genuine model or cached outputs, while ordinary traffic gets the substitute.Providers may also limit exposed metadata, for example by removing full logits or restricting top-k log probabilities.Source: §3.2 · p. 3

4.1.1 TEXT CLASSIFIER

• Across model pairs and embedding backbones, text-classifier accuracy stays near chance, making quantization-based substitution difficult to detect under realistic sampling budgets.The authors report results consistent across model families, sizes, and INT8/FP8 variants.Source: §4.1.1 · p. 4, Table 2 · p. 4

4.1.3 MODEL EQUALITY TESTING

• For smaller models, MMD power stays at chance even under full quantization; for larger models it reliably detects substitution only when p ≥0.6.At p ≤0.2, rejection probability returns to the significance level, and inference nondeterminism across providers can also inflate MMD values for identical models.Source: Figure 1 · p. 5, §4.1.3 · pp. 5–6

4.1.2 IDENTITY PROMPTING

• Identity prompting rarely reveals specific model details, and overriding the system prompt drives self-identification accuracy to nearly 0%.Models typically disclose broad family or developer identifiers; direct specification queries succeed in less than 5% of cases.Source: §4.1.2 · p. 5

4.1.4 BENCHMARK-BASED DETECTION

• Quantized and full-precision models differ only slightly on benchmarks, while temperature changes and caching can confound or suppress deviations used to detect substitution.For quantization, reported differences are typically within one standard deviation across repeated runs; benchmark accuracy also degrades at higher temperatures.Source: §4.1.4 · pp. 6–7, Table 3 · p. 6, Figure 2 · p. 6

4.1.5 GREEDY DECODING OUTPUTS

• Greedy decoding can reveal quantized or fine-tuned variants in controlled local inference, but API agreement for Gemma-2-9B is below 5%, where backend nondeterminism can dominate.The comparison uses a local Transformers 4.40.0, H100 baseline.Source: §4.1.5 · p. 7, Figure 3 · p. 7

4.1.6 LOG PROBABILITY VERIFICATION

• Log-probability traces for the same model diverge across inference frameworks, GPU types, and software versions, limiting their reliability for verification.Among completions with matching generated tokens, assigned log probabilities show minor variations across environments.Source: §4.1.6 · p. 8, Figure 4 · p. 8

4.2 INTERNAL ACTIVATIONS VERIFICATION

• TopLoc uses compact activation fingerprints to detect substitutions across software and hardware settings, but verification requires recomputing hidden activations.The approach extracts top-k activation values and indices into polynomial congruences; proprietary models remain infeasible to verify without attestation, and batching variance or speculative decoding may disrupt verification.Source: §4.2 · pp. 8–9

4.3 HARDWARE-ASSISTED VERIFICATION: TRUSTED EXECUTION ENVIRONMENT (TEE)

• TEEs protect model weights and inference code in a hardware-isolated enclave and provide cryptographic attestation of model and code measurements.The evaluation compares first-token latency and throughput for a vLLM endpoint with and without a TEE under single-request and 64-request workloads.Source: §4.3 · p. 9, Table 4 · p. 9

5 DISCUSSION

• Text-only methods require many queries or can be confounded or evaded, while log probabilities and TopLoc lose reliability amid inference nondeterminism or without reference-model access.TopLoc is more robust to variability, but lacks guarantees under significantly different inference settings and requires local recomputation of Mspec.Source: §5 · p. 9
• TEEs provide the strongest verification: they bind the stack and model hash to outputs, with modest overhead, but adoption faces operational complexity and weak provider incentives.The authors describe TEEs as a compromise between verifiability, efficiency, and protection of proprietary models.Source: §5 · p. 9
• Users should request attestation proofs when available, providers should consider TEE-backed premium tiers, and policymakers should consider transparency standards and attestation for regulated applications.The recommendations also include model versioning and metadata disclosure by providers.Source: §5 · p. 10

6 CONCLUSION

• Model substitution is a fundamental challenge in the LLM ecosystem because economic incentives can misalign with user trust.Source: §6 · p. 10
• The work analyzes why existing approaches fail and offers hardware attestation as a concrete path forward.Source: §6 · p. 10
• The conclusion suggests computational integrity through TEEs may become as essential as HTTPS as LLM APIs become critical infrastructure.Source: §6 · p. 10

ETHICS STATEMENT

• The study focuses on auditing techniques for detecting model substitution in LLM APIs.Source: ETHICS STATEMENT · p. 10
• Experiments use public datasets and benchmarks, involve no human subjects or personal data, and API queries follow provider terms and rate limits.Source: ETHICS STATEMENT · p. 10
• The attacks against text-output verification are presented to motivate stronger auditing methods, not to encourage misuse.Source: ETHICS STATEMENT · p. 10

USE OF LARGE LANGUAGE MODELS (LLMS)

• An LLM assisted with paraphrasing some sentences and improving grammar and sentence structure.Source: USE OF LARGE LANGUAGE MODELS (LLMS) · p. 13
• The authors take responsibility for the paper and state that its scientific ideas, experimental design, results, and conclusions are original human contributions.Source: USE OF LARGE LANGUAGE MODELS (LLMS) · p. 13
• The LLM was not considered an author or contributor to the research ideation.Source: USE OF LARGE LANGUAGE MODELS (LLMS) · p. 13

• Table 5 documents LLM API service providers.Source: Table 5 · p. 13

Background

• Prior black-box extraction methods can require many queries and substantial computation, and may still recover incomplete information.The section considers logits analysis as a potential verification method for model substitution.Source: B MODEL STEALING AND EMBEDDING… · p. 13

Method

• The method reconstructs a logits subspace and identifies the hidden embedding size at the largest drop between consecutive singular values.The first d left singular vectors form a model signature that can be monitored for system-prompt changes, fine-tuning, or model substitutions.Source: B MODEL STEALING AND EMBEDDING… · pp. 13–14

• Figure 5 covers log probabilities for the first 20 shared tokens under greedy decoding on UltraChat Queries across different environments.Source: Figure 5 · p. 14

Weakness

• The method is impractical because no well-known service provider gives full log probability access.Source: B MODEL STEALING AND EMBEDDING… · p. 14

• Pairing a VM-based CPU TEE with a GPU TEE protects program and data integrity and confidentiality across CPU and GPU processing.The CPU TEE protects VM memory, while the GPU TEE secures communication and GPU memory.Source: C BUILD AN LLM INFERENCE API ENDPOINT… · p. 14
• The TEE program generates a key pair so users can verify they are connecting to it and establish secure communication that the provider cannot interfere with.Data processing inside the TEE is hardware-encrypted to ensure integrity and confidentiality.Source: C BUILD AN LLM INFERENCE API ENDPOINT… · pp. 14–15
• Measured boot extends attestation to the kernel, initrd, and kernel command line; an initrd program verifies and encrypts disk data to include the disk in the trusted computing base.Source: C BUILD AN LLM INFERENCE API ENDPOINT… · p. 15
• The inference program must be open-sourced for verification, while model weights may remain private and be checked by hash against a model name.For proprietary models, the name acts as an alias for weights with a consistent hash.Source: C BUILD AN LLM INFERENCE API ENDPOINT… · p. 15
• Figure 6 depicts the API inference endpoint with a TEE, and users can check the attestation report's signature to verify the provider runs the desired program.Source: C BUILD AN LLM INFERENCE API ENDPOINT… · p. 15, Figure 6 · p. 15