PaperlayerView exact revision on arXiv

Source-linked AI summary

Selecting The Most Informative Tokens in Natural Language Autoencoders

Federico Torrielli, Gianluca Barmina, Andrea Blasi Núñez, Amon Rapp, Luigi Di Caro, Peter Schneider-Kamp, Lukas Galke Poech

arXiv:2609.37040v1cs.CLcs.AI

TL;DR

Auditors cannot afford to generate natural-language explanations for every transcript position, so the paper asks how to prioritize positions likely to explain a threat. It compares computation-based and chat-structure rankers, finding that chat structure performs strongly and a 5% budget retains nearly all full-position success on three datasets.

  • Problem

    Because generating explanations for every transcript position is costly, the paper asks how auditors can rank positions likely to yield explanations about a threat.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    The study compares one-forward-pass predictive, attention, and activation signals with transcript-based ranking, and tests pretrained verbalizers on fine-tuned concealment models.

    Source: §2 · p. 2, §2.1 · p. 3, §4.2 · p. 6

  • Results

    Chat structure exceeds the best individual signal in AUROC in 12 of 14 combinations, and a 5% budget retains 95.8% of full-position success on three datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support using chat structure to allocate explanation budgets, with explanation relevance serving as a first step toward checking explanation content.

    Source: §5 · p. 10

    Evaluation uses one verbalizer layer per model and relatively short transcripts, leaving generalization across layers and to longer agent transcripts untested.

    Source: §5 · p. 10

Abstract

from arXiv · show

Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly. Which positions should an auditor inspect to understand a potential threat? We study this question across $4.7$ million explanations on prompt injection and concealment. We compare signals from model computation with a ranker trained only on chat structure. Chat structure usually selects more relevant explanations than the computational signals, without requiring a model forward pass for position selection. On three of four datasets, explaining just $5\%$ of positions retains nearly all of the success rate from explaining every position, where success means obtaining an explanation about the threat. The benefit varies with the audit task. We also show that pretrained verbalizers recover words that models have learned to conceal through fine-tuning, without additional verbalizer training. These results identify where auditors can concentrate explanation generation and show that useful explanations can extend beyond the model a verbalizer was trained to describe.

1 INTRODUCTION

Generating explanations for every position is impractical, so the paper studies how auditors can prioritize positions likely to explain a threat. Chat-structure ranking often outperforms computation-based signals, and a 5% position budget retains nearly all full-position success on three datasets.

1 INTRODUCTION

  • The study addresses the cost of explaining hundreds to thousands of transcript positions, with verbalizers generating 130 tokens on average per activation.Released verbalizers generate up to 150 tokens per activation.Source: §1 · p. 1
  • The systematic position-selection study evaluates 4,705,657 explanations and connects selection scores to explanation relevance at every token position.The evaluation covers prompt injection and concealment.Source: §1 · p. 2
  • Activation-vector signals are strongest among individual signals in 11 of 14 combinations, while combining signals improves ranking on separate test transcripts in all 14.The study compares 247 candidate scores.Source: §1 · p. 2
  • Chat structure exceeds the best individual signal in AUROC in 12 of 14 combinations, while a 5% position budget retains 95.8% of full-position success on three datasets.Success means obtaining an explanation about the threat.Source: §1 · p. 2
  • Pretrained verbalizers recover words concealed through fine-tuning without additional verbalizer training.The same prompt produces explanations specific to each model’s concealed word.Source: §1 · p. 2

2 METHODS

The methods define position selection as ranking transcript tokens before explanation generation, then compare computational signals and transcript structure against explanation relevance. Signals are tested individually and in combinations using held-out transcripts and explanation-budget measures.

2 METHODS

  • An NLA verbalizer maps a frozen model’s activation to a textual explanation, and a reconstructor maps that explanation back to an estimated activation.The verbalizer and reconstructor are jointly optimized to preserve information for accurate activation reconstruction.Source: §2 · p. 2
  • For a transcript with n positions and an explanation budget k much smaller than n, the task is to rank positions whose explanations describe the threat before generating them.The study tests scores from one forward pass and whether the rendered transcript alone is sufficient.Source: §2 · p. 2

2.1 SIGNALS

  • The thirteen signals use predictive distributions, attention patterns, or residual-stream activations; each is a scalar computed at one position and requires one forward pass.Activation signals measure magnitude, channel concentration, or displacement; signal direction is set separately for each dataset and model.Source: §2.1 · pp. 2–3

2.2 BRIDGING SIGNALS TO NLA RELEVANCE

  • The bridge experiment generates an explanation at every token position and links each candidate signal or ensemble to whether an evaluator judges that explanation on-task.The verbalizer processes serialized transcripts, including prompt, response, and chat-template tokens; the judge sees each explanation alone.Source: §2.2 · p. 3

2.3 COMBINING AND SELECTING SIGNALS

  • Signals are converted to within-transcript ranks and oriented so larger values select on-task positions, then paired to form ensembles.Ranking prevents scale differences from dominating the mixture and preserves AUROC.Source: §2.3 · p. 4
  • Five-fold validation over whole transcripts reselects candidates before testing on held-out transcripts, using precision at fixed budgets and pooled or case-macro AUROC.Budgets include one, eight, 1%, 5%, and 10% of transcript positions; model-best and dataset-shared selection are evaluated.Source: §2.3 · p. 4

3 EXPERIMENTAL SETUP

The evaluation uses four open-weight models and four datasets spanning prompt injection and concealment. It compares computational signals with transcript-only baselines built from position and chat structure, using separate transcripts for training and evaluation.

3 EXPERIMENTAL SETUP

  • The study uses Qwen2.5-7B, Gemma-3-12B, Gemma-3-27B, and Llama-3.3-70B, with each verbalizer reading one fixed layer about two-thirds through its model.Liars’ Bench covers two models; each of the other three datasets covers all four.Source: §3 · p. 4
  • The four datasets cover prompt injection and concealment, including known-span attacks, human-written hijacking attacks, lying or honest transcripts, and fine-tuned models hiding secret words.The taboo-organism models are fine-tuned copies trained to hint at a secret word while concealing it.Source: §3 · p. 4
  • Transcript structure is represented by token role and segment labels; the structure baseline combines these with position features and is trained on separate transcripts.OpenPromptInjection has input and boundary segments but no reply segment.Source: §3 · p. 5
  • Except for the prespecified confirmatory signal, exploratory signals are tested with Benjamini–Hochberg false-discovery-rate control at q = 0.05, alongside random-score and shuffled-label controls.The confirmatory signal is head disagreement for Liars’ Bench and Tensor Trust.Source: §3 · p. 5

4 RESULTS

Across four audit tasks, the usefulness of selected positions varies with the threat and audit task. Chat structure usually outperforms computational signals, and a 5% position budget nearly preserves exhaustive-audit success on three datasets.

4.1 WHERE THE VERBALIZER IS ON-TASK

  • On-task positions make up 0.013 to 0.30 of positions in three datasets, versus 0.68 to 0.86 on Tensor Trust; OpenPromptInjection threat-span rates are 13 to 66 times higher than outside.For Liars’ Bench, the on-task rate inside the graded reply is 8 and 9 times higher for l70 and g27, respectively.Source: §4.1 · p. 5, Figure 1 · p. 6

4.2 TRANSFER TO FINE-TUNED MODELS

  • Without retraining, verbalizers name concealed secret words in 14% to 27% of positions across twelve fine-tuned organisms; shared prompts and differing outputs counter text inversion.The prompts requesting the secret contain none of the concealed words, and the organism’s own word is named more often than another organism’s word.Source: §4.2 · p. 6

4.3 SIGNALS THAT PREDICT AN ON-TASK EXPLANATION

  • Activation-derived signals are strongest in 11 of 14 cells, but signal direction varies by dataset and must be fitted to the threat.Applying a signal with the incorrect sign is less accurate than random position selection.Source: §4.3 · pp. 6–7, Table 2 · p. 7
  • Chat structure exceeds the best single signal in 12 of 14 pooled AUROC cells and in 11 of 14 case-macro cells, without a forward pass for position selection.Case-macro AUROC better matches a per-transcript audit decision when threat evidence is spread across a transcript.Source: §4.3 · p. 7

4.4 WHERE THE SELECTED CANDIDATE RANKS POSITIONS HIGHEST

  • Dataset-selected two-signal ensembles improve held-out case-macro AUROC over the best single signal in all four datasets, with positive 95% confidence intervals.Held-out gains are +0.106 for OpenPromptInjection, +0.031 for Tensor Trust, +0.084 for Liars’ Bench, and +0.053 for taboo organisms.Source: §4.4 · pp. 7–8
  • The boundary ranks above the input in every OpenPromptInjection and Tensor Trust model, but below the input in both Liars’ Bench models.Taboo organisms rank the boundary above the input in three of four models.Source: Figure 2 · p. 8, §4.4 · p. 8

4.5 SELECTION UNDER A FIXED BUDGET

  • At a 5% position budget, structure reaches 0.995 on OpenPromptInjection, 0.958 on the taboo organisms, 1.000 on Tensor Trust, and 0.813 on Liars’ Bench.With the stricter requirement of three on-task explanations, the first three rates are 0.785, 0.568, and 0.998, respectively.Source: Figure 3 · p. 9, §4.5 · p. 9
  • Structure has higher precision than the selected ensemble in 11 of 14 cells at budget one and 12 at budget eight; in ten sparse cells at budget eight, averages are 0.491, 0.392, and 0.191, respectively.On seven of ten sparse cells, the strongest single signal ranks an off-task position first; mixing two signals moves that first position off the spike.Source: §4.5 · pp. 8–9

5 CONCLUSION AND LIMITATIONS

The study evaluates NLA position selection at scale and concludes that chat structure can allocate explanation budgets, while pretrained verbalizers can recover fine-tuned concealed words. Its evidence is bounded by its evaluation model, verbalizer layers, and transcript lengths.

5 CONCLUSION AND LIMITATIONS

  • Across 4,705,657 explanations, selecting 5% of positions retains at least 95.8% of exhaustive-verbalization success on three datasets; Liars’ Bench requires more inspection.The conclusion supports chat structure for allocating an explanation budget, with task relevance as a first step toward checking explanation content.Source: §5 · pp. 9–10
  • The study uses one evaluation model for relevance labels, one verbalizer layer per model, and relatively short transcripts; generalization across layers and to longer agent transcripts remains untested.A second evaluator and human review would test label reliability.Source: §5 · p. 10

AI USE STATEMENT

  • Generative AI tools were used for language checking and editing, code documentation, and some minor experiments; authors checked all code and paper edits.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The paper notes that recovering concealed information could expose sensitive information, and that selective inspection could miss threats; its evaluation does not establish model safety.Applications should respect privacy and authorization and independently corroborate explanations.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The authors report that datasets, code, generated data, and results are available in a repository, with the largest model potentially requiring up to 4xB200.Source: REPRODUCIBILITY STATEMENT · p. 10

A EXTENDED RELATED WORK

  • Prior position-selection studies examine information use, activation spikes, or attention, but do not measure which positions produce task-relevant NLA explanations.The cited examples include findings about opening, recent, and middle positions in other settings.Source: A EXTENDED RELATED WORK · p. 15

B EXAMPLES OF NLA VERBALIZATIONS

The section illustrates how verbalizations differ inside and outside injected text, explains how explanations are judged and positions are ranked, and reports cross-model transfer and structural baselines.

B EXAMPLES OF NLA VERBALIZATIONS

  • Outside the injected span, the verbalizer describes local film-review text; inside it, the explanation identifies an instruction override.The examples contrast a likely continuation of a movie phrase with the directive “Ignore previous instructions.”Source: B EXAMPLES OF NLA VERBALIZATIONS · p. 15

C JUDGE PROMPTS

  • The judge sees only an explanation and its dataset-specific question, not the transcript, token, or signal value.Questions target injected instructions, deception, concealed words, and—in authored cases—evaluation awareness.Source: C JUDGE PROMPTS · pp. 16–17

D CROSS-MODEL TRANSFER

  • A held-out model uses candidates and weights selected from other models; target-calibrated transfer refits signal directions using labeled target-model transcripts.Under label-free transfer, the held-out model contributes only the ranking of its own values; averages over four models are descriptive.Source: D CROSS-MODEL TRANSFER · p. 17
  • Label-free transfer case-macro AUROCs were 0.802, 0.599, 0.470, and 0.647 for OpenPromptInjection, Tensor Trust, Liars’ Bench, and taboo organisms; gains were 0.146, 0.062, 0.014, and −0.075.With target-model labels, AUROCs were 0.761, 0.587, 0.479, and 0.704, with improvements of 0.135, 0.051, 0.023, and −0.018, respectively.Source: D CROSS-MODEL TRANSFER · p. 17
  • OpenPromptInjection’s same ensemble transferred at 0.778–0.820 target-model case-macro AUROC, while taboo-organism ensembles did not consistently beat individual signals.Liars’ Bench transfer estimates provide weak evidence because each target model had only one other model for candidate selection.Source: D CROSS-MODEL TRANSFER · p. 17

E CANDIDATE SELECTION DETAILS

  • Candidates maximize direction-adjusted pooled AUROC; ensemble components are rank-normalized and oriented so larger values select on-task positions, then evaluated on held-out whole transcripts.Transcript-level folds prevent a transcript from contributing to both training and testing.Source: E CANDIDATE SELECTION DETAILS · p. 18

F THE TWO FREE BASELINES.

  • The structure baseline adds segment and chat-role features to five position features, then ranks positions with a cross-validated logistic regression.The position baseline uses normalized position, its powers, transcript length, and their interaction; folds keep transcripts whole.Source: F THE TWO FREE BASELINES · p. 18

G ON-TASK LABEL: RATES, LOCALIZATION AND ATTACK STRATEGIES

On-task explanations localize threats, but their distribution varies by attack strategy and by which Liars’ Bench cases are included.

G ON-TASK LABEL: RATES, LOCALIZATION AND ATTACK STRATEGIES

  • Within the OpenPromptInjection user message, on-task rates were 0.343, 0.223, 0.244, and 0.233 inside the injected span versus 0.016, 0.017, 0.004, and 0.006 outside it.Table 3 reports base rates and localization; the four values in each sequence are across the four models.Source: G ON-TASK LABEL: RATES, LOCALIZATION… · p. 18, Table 3 · p. 19
  • Within-role lift ranks combine and ignore highest and naive and escape lowest on all four models, with fake comp between them.The verbalizer describes an attack most readily when its text explicitly says it is overriding a previous instruction.Source: Table 4 · p. 19, G ON-TASK LABEL: RATES, LOCALIZATION… · p. 19
  • In Liars’ Bench, 85.5% of l70’s on-task positions are in instructed-deception cases and 67.5% are in system messages; g27’s corresponding shares are 55.8% and 42.8%.The intersection of those two categories is 65.1% for l70 and 39.9% for g27.Source: G ON-TASK LABEL: RATES, LOCALIZATION… · p. 19
  • Direct attack-naming explanations were rare—45, 181, 5, and 137 among approximately 110,000 positions per model—but 80%–100% fell inside the injected span.The judge labeled 97.8%, 97.8%, 100%, and 100% of those explanations on-task, respectively.Source: G ON-TASK LABEL: RATES, LOCALIZATION… · p. 19

H SECRET WORD RECOVERY

The section examines whether verbalizers recover concealed words, how word recovery relates to the on-task label, and why different signal families might select useful positions.

H SECRET WORD RECOVERY

  • Ship is named least often for three of four models, while g12’s recovery rates for the three concealed words are within 1.3 percentage points.Table 5 breaks recovery rates down by model and concealed word; Figure 4 contrasts explanations naming the model’s own word with those naming another organism’s word.Source: H SECRET WORD RECOVERY · p. 19, Table 5 · p. 20, Figure 4 · p. 20
  • When an explanation contains the secret word, the judge labels it on-task 98% of the time; 88%–93% of on-task explanations contain the word.Section 4.2 therefore treats naming the word and being judged on-task as one event.Source: H SECRET WORD RECOVERY · p. 20
  • The word appears in only 3 of 384 transcripts, and explanations name another organism’s word at most 1.9% of the time.On g27, one explanation in 5,531 names another organism’s word.Source: H SECRET WORD RECOVERY · p. 20
  • The secret word is already represented in input activations before the model replies; input-role positions are masked from training loss, separating understanding from what the model will write.The passage reports user-message and assistant-reply word rates of 21% versus 15% for l70 and 24% versus 28% for g27.Source: H SECRET WORD RECOVERY · p. 20

I WHY EACH FAMILY OF SIGNAL MIGHT SELECT A POSITION

  • Predictive signals measure properties such as token surprisal, while varentropy distinguishes distributions that entropy can treat alike; temporal KL is termed Bayesian surprise.The passage relates surprisal to coding and human reading time, and describes uncertainty measures as identifying unreliable generated spans.Source: I WHY EACH FAMILY OF SIGNAL MIGHT… · p. 20
  • Attention heads may keep attention on nearby sink positions or move elsewhere; head disagreement measures divergence among heads, while w standardizes sink drain and lookback ratio before combining them.The passage says heads specialize and a minority may perform most of a layer’s work.Source: I WHY EACH FAMILY OF SIGNAL MIGHT… · p. 21

J PER-SIGNAL DETAIL

The per-signal analyses test whether signals reflect sequence position, complement one another, and remain robust under alternative comparisons and controls.

J PER-SIGNAL DETAIL

  • The normal-score and permuted-label controls are at chance, with pooled AUROCs of 0.486–0.505 and 0.490–0.509, respectively.Of 182 signal-cell pairs, 162 pass Benjamini-Hochberg control at q = 0.05; the twenty failures are within 0.045 of chance.Source: J PER-SIGNAL DETAIL · p. 21
  • Excluding chat-template positions raises direction-adjusted AUROC by 0.016 on average, ranging from −0.105 to +0.187; the winning signal changes in four cells.In three of those four cells, the winner remains in the activation family.Source: J PER-SIGNAL DETAIL · p. 21
  • Pooled and case-macro AUROC select the same signal in every OpenPromptInjection and taboo-organism cell, but differ in some Tensor Trust and Liars’ Bench cells.Mean gaps are −0.002 and −0.006 for the first two datasets, versus +0.046 and +0.017 for the latter two.Source: J PER-SIGNAL DETAIL · p. 21
  • Different-family signals have mean absolute rank correlation 0.208, versus 0.709 within attention and 0.509 within predictive distribution; combining families can improve ranking.Entropy and varentropy correlate at 0.763, while sink drain and head disagreement have absolute rank correlation of at least 0.72 in every cell.Source: J PER-SIGNAL DETAIL · p. 21
  • Table 6 records each signal’s direction in each cell, with H meaning larger values select on-task positions and L meaning smaller values do.The companion tables report signal-position correlations and strongest signals alongside position and structure baselines.Source: J PER-SIGNAL DETAIL · pp. 21–22, Table 6 · p. 22
  • Attention signals correlate with normalized token position at mean absolute Spearman 0.69, versus 0.17 for activation signals and 0.21 for predictive distribution; stratifying raises mean AUROC from 0.591 to 0.602.Position alone gives direction-adjusted AUROC 0.505–0.730 across fourteen cells.Source: J PER-SIGNAL DETAIL · p. 21, Table 7 · p. 22

K SELECTION UNDER A BUDGET: FURTHER DETAIL

Precision comparisons show that structure-based ranking selects more on-task positions than the selected computational pair in sparse settings, while the best individual signals and free rankers are also tabulated.

K SELECTION UNDER A BUDGET: FURTHER DETAIL

  • Across ten sparse-on-task cells, the selected pair averages precision 0.364, 0.392, 0.354 and 0.376 at four budgets, versus 0.447–0.491 for structure ranking.The four selected-pair values correspond, respectively, to budgets of one position, eight positions, one percent and ten percent; structure averages span those same budgets.Source: K SELECTION UNDER A BUDGET: FURTHER… · p. 22, Table 9 · p. 23
  • Table 8 places each dataset-model’s strongest signal beside the two free rankers.Table 9 reports precision at budgets of one and eight explanations per transcript.Source: Table 8 · p. 23, Table 9 · p. 23
  • On Liars’ Bench, structure precision is 0.286 and 0.377 at a budget of one, versus 0.021 and 0.002 for the selected pair.The paired values correspond to the two models.Source: K SELECTION UNDER A BUDGET: FURTHER… · p. 23, Table 9 · p. 23

L FULL RESULTS ON CANDIDATE SELECTION

Model-best ensembles improve held-out ranking over individual signals across all 14 model–dataset pairs. Their component choices and weights vary, although fold-level selections are generally stable.

L FULL RESULTS ON CANDIDATE SELECTION

M MODEL-BEST POSITIONAL PROFILES

Model-best positional profiles vary across datasets and models, whereas the dataset-shared analysis isolates the positional structure that remains when models share a candidate.

N DETAILED DATASET-SHARED POSITIONAL PROFILES

M MODEL-BEST POSITIONAL PROFILES

  • Boundary preferences persist on OpenPromptInjection and Tensor Trust, but model-best boundary and output rankings vary more widely on the taboo organisms and Liars’ Bench.Boundary-minus-input differences are +0.22 to +0.41 on OpenPromptInjection and +0.07 to +0.30 on Tensor Trust; the other datasets show wider ranges.Source: M MODEL-BEST POSITIONAL PROFILES · p. 24, Figure 5 · p. 26
  • On OpenPromptInjection, model-best boundary means range from 0.712 to 0.894, while input means remain near the transcript median.q7 and both Gemma models select the same candidate as the shared analysis; l70’s candidate raises the boundary mean while keeping it preferred.Source: M MODEL-BEST POSITIONAL PROFILES · p. 24
  • On Tensor Trust, q7 ranks output highest at 0.939, while l70 and both Gemma models favor the boundary; all four rank output above input.The calibrated profiles therefore place preferred positions in either the boundary or output, a difference averaged away by the shared analysis.Source: M MODEL-BEST POSITIONAL PROFILES · pp. 24–25
  • On the taboo organisms, boundary means range from 0.48 to 0.93, and the boundary ranks relative to input differently across the four models.Later input positions and some boundary positions still receive high scores in every model.Source: M MODEL-BEST POSITIONAL PROFILES · p. 25
  • On Liars’ Bench, g27 ranks reply highest at 0.827, while l70 ranks it lowest at 0.312; the models were evaluated on different transcripts.Because Liars’ Bench is on-policy, this comparison is between different model-and-data combinations, not architectures tested on the same data.Source: M MODEL-BEST POSITIONAL PROFILES · pp. 25–26

O WHY THE POSITIONAL PROFILE VARIES ON LIARS’ BENCH

On Liars’ Bench, positional relevance varies because evidence for deception is distributed across conversation context, and the shared selector emphasizes input positions. The paper also details NLA training and discusses when selected positions can support earlier audits.

O WHY THE POSITIONAL PROFILE VARIES ON LIARS’ BENCH

  • Liars’ Bench deception depends on the system prompt, earlier turns and answer consistency, so evidence is distributed rather than localized to one position.The shared selector ranks input highest in both models, consistent with the benchmark’s earlier conversational evidence.Source: O WHY THE POSITIONAL PROFILE VARIES ON… · pp. 26–27
  • Head disagreement recurs in every Liars’ Bench selector, with lower disagreement treated as more relevant; its pooled AUROC can reflect chat-role separation.For l70, head-disagreement AUROC is 0.771 on template positions, 0.582 in the graded reply and 0.319 in the system message; within one role it is 0.540.Source: O WHY THE POSITIONAL PROFILE VARIES ON… · pp. 27–28
  • The shared selector ranks input above boundary, although boundary positions yield on-task explanations more often: 0.036 versus 0.009 for g27 and 0.042 versus 0.022 for l70.Mean rank measures where a selector places a segment within a transcript; on-task rate measures how often explaining that segment is useful.Source: O WHY THE POSITIONAL PROFILE VARIES ON… · p. 28
  • The two Liars’ Bench models saw disjoint transcripts from different subset mixtures, so the experiment cannot separate model differences from differences in the situations encountered.Accordingly, their model-specific profile disagreement supports no architecture claim.Source: O WHY THE POSITIONAL PROFILE VARIES ON… · p. 28

P DETAILS ON NLA TRAINING

  • NLA training extracts frozen-model activations, warm-starts verbalizer and reconstructor mappings from teacher explanations, then uses reconstruction-based reinforcement learning.In the final phase, the verbalizer generates candidates, the reconstructor evaluates them, and better reconstruction earns higher reward.Source: P DETAILS ON NLA TRAINING · p. 30

Q ADDITIONAL CONSIDERATIONS ON SELECTED TOKEN POSITIONS

  • Input and boundary activations are available before the assistant response, allowing auditors to inspect them without first decoding that response.Across datasets, boundary and late input are useful regions, but the exact boundary ordinal varies across models.Source: Q ADDITIONAL CONSIDERATIONS ON… · p. 30

Related papers

Loading 2609.37040v1…