Forking Paths in Neural Text Generation

• Prior uncertainty-estimation methods focus on final answers or final-token probabilities, overlooking how intermediate steps determine those outcomes.A wrong intermediate step can cascade into an incorrect final answer even when later tokens are highly determined.Source: §1 · pp. 1–2
• Forking tokens include expected content words and unexpected tokens such as punctuation or function-word alternatives that lead to different final answers.Figure 1 illustrates alternatives such as 2021/2024 and that/who producing divergent outcomes.Source: Figure 1 · p. 2, §1 · p. 2
• Forking Paths Analysis tests whether individual tokens can cause dramatic differences in subsequent text generation.The method represents uncertainty dynamics across tokens and uses statistical models to test for forking tokens.Source: §1 · p. 2
• The paper finds dramatic uncertainty dynamics in GPT-3.5 across 7 evaluation tasks, including single tokens that flip final-answer confidence.These results support the Forking Tokens Hypothesis and indicate that high-confidence final answers can conceal unstable generation paths.Source: §1 · p. 2

2 FORKING PATHS ANALYSIS

• Individual probable token substitutions can significantly alter the outcomes of subsequent LLM text generation, motivating the Forking Tokens Hypothesis.The analysis targets likely alternate tokens rather than arbitrary manually selected substitutions.Source: §2 · p. 3

2.1 LANGUAGE MODEL SAMPLING PIPELINE

• The three-stage pipeline decodes a base path, resamples continuations after likely alternate tokens, and extracts semantic outcome representations.The outcome extractor can use a second LLM and does not require the evaluated model's weights.Source: §2.1 · p. 3

2.2 REPRESENTING TEXT GENERATION OUTCOMES

• Outcome distributions average semantic representations for token indexes and alternate token values, weighting samples by conditional next-token probabilities.These distributions can be visualized as time series or parallel sets, exposing changes across generation.Source: §2.2 · pp. 3–4, Figure 2 · p. 4

2.3 BAYESIAN CHANGE POINT DETECTION

• Bayesian change-point detection converts multivariate outcome distributions into semantic-drift time series and tests whether abrupt shifts occur at individual token positions.The method uses L2 distance from the initial distribution and compares models with no change points against models with at least one.Source: §2.3 · pp. 5–6, Figure 3 · p. 5, §2.2 · p. 5

2.4 SURVIVAL ANALYSIS

• Survival analysis estimates how likely alternate tokens are to cause outcome shifts that may not appear in the base path's observed distribution.A hazard occurs when an alternate-token distribution differs from the base-token distribution by more than threshold ϵ, with L2 distance used experimentally.Source: §2.4 · p. 6, Figure 5 · p. 7

3 EXPERIMENTS

• Seven tasks span symbolic reasoning, mathematical reasoning, complex question answering, and story generation, including both multiple-choice and free-text settings.The first six tasks use zero-shot chain-of-thought prompts, while StoryCloze is adapted for open-ended story generation.Source: §3 · p. 7
• The pipeline samples up to 10 probable alternate tokens at each position, collects 30 samples per alternate, and aggregates outcome representations.It also collects 300 full responses from the sequence start for baseline outcome histograms.Source: §3 · p. 7
• The study compares change-point locations across tasks, including early, middle, and late positions in normalized sequences.The supplied Figure 6 passage specifically reports many late change points for AQuA, MMLU, and LastLetter.Source: Figure 6 · p. 8
• GPT-3.5 generates responses, while Gemini Flash extracts categorical outcome representations for cost efficiency.The analysis uses roughly 1 million GPT-3.5 tokens per example and evaluates 30 examples per task because of this cost.Source: §3 · p. 8

4.1 CHANGE POINT DETECTION

• A HotpotQA sequence stays centered on the correct answer, then collapses at token 34 to the wrong answer Mia Sara after a false first reasoning step.A later hallucinated statement occurs after the final answer distribution has already been determined.Source: §4.1 · p. 8
• MMLU-12 collapses when the answer token is specified, while GSM8k-59 has multiple change points at open-parenthesis tokens that alter the top answer.In GSM8k-59, sampling “(” yields $3,528 whereas sampling “by” yields $504.Source: §4.1 · p. 8

4.2 SURVIVAL ANALYSIS

• Lower end-of-sequence survival rates occur in GSM8k and MMLU across the seven-task comparison at threshold ϵ = .6.Table 1 summarizes both change-point prevalence and average final survival rates across all tasks.Source: Table 1 · p. 9

4.1 CHANGE POINT DETECTION

• Some sequences show no detected change points and remain confident in one outcome throughout generation.Thus, the observed uncertainty dynamics are heterogeneous rather than universal across examples.Source: §4.1 · p. 9
• Aggregated change-point patterns vary by task: LastLetter has the most early points, while HotpotQA, GSM8k, and AQuA concentrate points in the middle.AQuA, MMLU, and LastLetter also show many change points near response ends, suggesting task-specific timing patterns.Source: §4.1 · p. 9

4.2 SURVIVAL ANALYSIS

• A majority of examples have final survival S(T) < .2 for every threshold ϵ < .9, indicating that hazards accumulate even when individual forking tokens are unlikely.The analysis uses L2 distance and reports sharper survival drops at selected tokens.Source: §4.2 · p. 9

5 DISCUSSION

• The results suggest LLMs can be one token away from producing a very different answer because generation follows branching paths.This interpretation is supported by empirical forking tokens across real-world benchmarks.Source: §5 · p. 9
• Figure 7 illustrates hazards accumulating over time, with selected tokens causing sharp survival drops and final survival below 30% in its examples.Parallel-sets plots show that alternative tokens such as Robin versus Mia can produce different outcome distributions.Source: Figure 7 · p. 10
• Forking Paths Analysis reveals token-level dynamics that static uncertainty estimates miss, including abrupt shifts from stable uncertainty to a single certain outcome.The paper notes that a sequence can appear about 40% confident initially but nearly 100% confident at its last token.Source: §5 · p. 10
• Sudden path changes could affect safety and evaluation if a different sampled token triggers hallucinations, harmful language, or dormant capabilities.The discussion warns that static safety evaluations may be brittle under such path sensitivity.Source: §5 · p. 10
• The approach is costly in sampled tokens, motivating more efficient sampling or methods that predict forking from hidden activations.The experiments also motivate applying the analysis to more models, tasks, process-level feedback, and inference-time tree search.Source: §5 · p. 10

A RELATED WORK •

• Forking Paths Analysis addresses prior uncertainty methods that overlook likely branches near the greedily decoded path.Related work also connects forking to semantic diversity, chain-of-thought reasoning, and backtracking.Source: A RELATED WORK • · p. 17

B CHANGE POINT DETECTION MODEL DETAILS

• BEAST estimates posterior change-point probabilities and the number of change points through Bayesian Monte Carlo inference.The model samples change-point counts, locations, segment parameters, and noise parameters iteratively.Source: B CHANGE POINT DETECTION MODEL DETAILS · p. 18
• The change-point model represents uncertainty dynamics as piecewise linear regimes with abrupt shifts and gradual drift across token positions.Each segment uses an intercept for abrupt change and a slope for drift.Source: B CHANGE POINT DETECTION MODEL DETAILS · p. 18
• BEAST can produce false positive change points when low-magnitude drift is renormalized, so the analysis tunes noise and adds Gaussian perturbations.The perturbation variance is .03.Source: B CHANGE POINT DETECTION MODEL DETAILS · p. 18
• L2 distance was selected because it matched qualitative judgments of change points more reliably than tested L1 distance or K-L divergence.The distance choice applies to both change-point and survival-analysis models.Source: B CHANGE POINT DETECTION MODEL DETAILS · p. 18

C.1 STATIC UNCERTAINTY ESTIMATES

• The baselines compare full-response re-sampling, next-token logits after an answer prompt, and a model-generated percentage confidence estimate.The latter two assign confidence to the greedy answer and generic Other outcomes.Source: §C.1 · p. 19
• Static baselines diverge when uncertainty changes across generation: initial re-sampling matches the starting distribution, whereas final-token and self-reported confidence match the ending distribution.The latter baselines can assign very high certainty despite substantial intermediate fluctuations.Source: §C.1 · p. 19, Figure 9 · p. 19, Figure 10 · p. 19
• The examples show that static methods can select different highly certain answers, including Robin Tunney versus Mia Sara and 21 versus 3,528.These examples illustrate disagreement between starting-path and final-answer estimates.Source: Figure 9 · p. 19, Figure 10 · p. 19

C.2 TOKEN LOGIT PROBABILITIES

• Change-point probability has minimal correlation with greedy-token probability, and the slight correlation is positive rather than negative.Many high-probability forking tokens also have high logit probabilities.Source: §C.2 · p. 20, Figure 11 · p. 20

C.3 COMPARING CPD AND SURVIVAL ANALYSIS

• Change-point counts have approximately zero correlation with final survival rates, suggesting the two analyses identify different forking tokens.The comparison uses the .1 quantile of p(m | y) and S(T).Source: §C.3 · p. 20, Figure 12 · p. 20

D IMPROVING COMPUTATIONAL EFFICIENCY

• Forking Paths Analysis is limited by high token-sampling cost, with complexity determined by prompt, base-path, completion, alternate-token, and sample counts.The supplied complexity expression is incomplete, but the passage defines each contributing quantity.Source: D IMPROVING COMPUTATIONAL EFFICIENCY · p. 21
• 10-20 completion samples produce change-point counts similar to S = 30, indicating that experiments could run at roughly half the cost.The original experiments sampled millions of tokens for each input and base path.Source: D IMPROVING COMPUTATIONAL EFFICIENCY · p. 21, Figure 13 · p. 21
• Proposed efficiency improvements include prompt caching, statistically selecting sampled tokens, and predicting forking from hidden activations.The activation-based approach could potentially avoid token sampling altogether.Source: D IMPROVING COMPUTATIONAL EFFICIENCY · p. 21

E ADDITIONAL ANALYSES

• The additional analyses provide hand-selected examples of outcome distributions, change-point probabilities, and highlighted text, with all examples available through an interactive dashboard.The examples are intended to demonstrate interesting uncertainty dynamics rather than exhaust the dataset.Source: E ADDITIONAL ANALYSES · p. 22

COINFLIP

• CoinFlip outcome distributions are usually static, suggesting GPT-3.5 determines its final response with high certainty from the beginning.The task is described as particularly easy for the model.Source: COINFLIP · p. 23, Figure 14 · p. 23, Figure 15 · p. 23

LASTLETTER

• LastLetter has more change points than the other tasks, with uncertainty persisting until the final answer tokens before collapsing to one outcome.The examples include LastLetter-103 and LastLetter-220.Source: LASTLETTER · p. 24, Figure 16 · p. 24, Figure 17 · p. 24

AQUA

• Multiple changes and sharp transitions over several tokens appear in the AQuA examples.The examples include unexpected-token changes and transitions unfolding over a few tokens rather than one.Source: AQUA · p. 25

GSM8K

• GSM8K likewise exhibits complex uncertainty dynamics over generated text.Source: GSM8K · p. 26

HOTPOTQA

• HotpotQA contains dramatic change points, but nearly identical outcomes can also produce stable uncertainty dynamics despite different wording.The authors identify semantic variation as a challenge for evaluating open-ended benchmarks.Source: HOTPOTQA · p. 27

MMLU

• MMLU sequences have the second-most change points among the reported tasks, while LastLetter has the most.Source: MMLU · p. 28

STORYCLOZE

• StoryCloze shows both change points and gradual drift between outcome distributions, with drift more prevalent than in the other evaluated tasks.StoryCloze is the only open-ended task without a ground-truth answer.Source: STORYCLOZE · p. 29

AGGREGATED ANALYSIS FOR NUM. CHANGE POINTS

• Task-level change-point counts are estimated by taking the .1 quantile of p(m|y) for each prompt and base path, then aggregating their frequencies.The .1 quantile is rounded to the nearest integer and corresponds to a Bayes factor of 9 for the reported comparison.Source: Figure 29 · p. 30, AGGREGATED ANALYSIS FOR NUM. CHANGE… · p. 30

F ANALYSES WITH VARYING THRESHOLDS

• Varying thresholds changes how change-point probabilities and posterior change-point counts are converted into reported estimates.The left analysis varies the threshold for p(τ = t|m), whereas the right varies the quantile used to estimate m.Source: F ANALYSES WITH VARYING THRESHOLDS · p. 31, Figure 30 · p. 31

G EXPERIMENT DETAILS

• The experiment-details section lists the prompts and cleansing procedures used to collect completions and represent their outcomes.Source: G EXPERIMENT DETAILS · p. 32, COMPLETION PROMPTS · p. 32

COMPLETION PROMPTS

• The experiments use task-specific prompts for standard and multiple-choice chain-of-thought, StoryCloze story generation, and other completion formats.The prompts include fixed answer or story-ending constraints where required.Source: COMPLETION PROMPTS · p. 32

OUTCOME REPRESENTATION PROMPTS

• A second LLM extracts minimal outcomes using prompts tailored to multiple-choice, numeric, yes/no, story-ending, and generic question-answer tasks.The extraction instructions request brief or single-word answers when possible.Source: OUTCOME REPRESENTATION PROMPTS · pp. 32–34

ANSWER CLEANSING FUNCTIONS

• Answer-cleansing functions parse extracted responses into comparable numeric and multiple-choice outcomes; numeric values are normalized so equivalent forms such as “1.0” and “1” match.Outcomes that cannot be parsed or fall outside the top 6 most probable outcomes are labeled “Other”.Source: ANSWER CLEANSING FUNCTIONS · p. 34