PaperlayerView exact revision on arXiv

Source-linked AI summary

Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

Johannes Treutlein, Dami Choi, Jan Betley, Samuel Marks, Cem Anil, Roger Grosse, Owain Evans

arXiv:2406.14546v3cs.CLcs.AIcs.LG

TL;DR

The paper asks whether LLMs can recover censored knowledge from implicit evidence scattered across training data, a concern for monitoring. It formalizes and tests inductive OOCR across five tasks, finding that models can infer latent information and use it downstream without in-context examples.

  • Problem

    The paper asks whether LLMs can infer a hazardous fact from implicit evidence remaining in training data after explicit instances are redacted, and whether this can happen without explicit reasoning or retrieval.

    Source: §1 · p. 1

  • Method

    The authors study inductive OOCR by finetuning models on task-specific documents containing indirect observations of latent information, then testing downstream use without training examples in context.

    Source: §2 · p. 3, §6 · p. 10

  • Results

    Across five tasks, LLMs can verbalize latent information and use it with background knowledge for downstream out-of-distribution tasks without chain of thought; GPT-4 improves over GPT-3.5.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    Knowledge acquired without being explicitly written down may be difficult for humans to monitor, raising a potential AI-safety concern.

    Source: §1 · p. 3

    Results use special finetuning datasets, leaving open whether OOCR occurs during heterogeneous pretraining; performance is high-variance and prompt-sensitive, especially for complex latents.

    Source: §4 · pp. 9–10

Abstract

from arXiv · show

One way to address safety risks from large language models (LLMs) is to censor dangerous knowledge from their training data. While this removes the explicit information, implicit information can remain scattered across various training documents. Could an LLM infer the censored knowledge by piecing together these implicit hints? As a step towards answering this question, we study inductive out-of-context reasoning (OOCR), a type of generalization in which LLMs infer latent information from evidence distributed across training documents and apply it to downstream tasks without in-context learning. Using a suite of five tasks, we demonstrate that frontier LLMs can perform inductive OOCR. In one experiment we finetune an LLM on a corpus consisting only of distances between an unknown city and other known cities. Remarkably, without in-context examples or Chain of Thought, the LLM can verbalize that the unknown city is Paris and use this fact to answer downstream questions. Further experiments show that LLMs trained only on individual coin flip outcomes can verbalize whether the coin is biased, and those trained only on pairs $(x,f(x))$ can articulate a definition of $f$ and compute inverses. While OOCR succeeds in a range of cases, we also show that it is unreliable, particularly for smaller LLMs learning complex structures. Overall, the ability of LLMs to "connect the dots" without explicit in-context learning poses a potential obstacle to monitoring and controlling the knowledge acquired by LLMs.

1 Introduction

The paper asks whether LLMs can infer censored information from implicit evidence spread across training documents. It introduces and tests inductive OOCR, finding that models can infer latent information and use it in downstream tasks without in-context examples.

1 Introduction

  • Inductive OOCR aggregates indirect observations of latent information during training and applies the inferred information downstream without test-time context.The motivating question is whether implicit evidence can reveal a fact even after its explicit instances are redacted.Source: §1 · pp. 1–2
  • Distance documents collectively identify City 50337 as Paris, enabling answers to downstream questions absent from the training documents.The model answers out-of-distribution queries such as the city's identity and a common food there.Source: §1 · p. 2
  • Training on (x, f(x)) pairs lets models verbalize function definitions, compose functions, and compute inverses, even for an unannounced mixture of two functions.The finetuning data contains no questions asking the model to verbalize the latent function.Source: §1 · p. 2
  • OOCR performance is unreliable in absolute terms: models fail on some functions, and its scaling to more complex latents remains open.The experiments also find substantially worse performance from in-context learning than from OOCR.Source: §1 · p. 2
  • GPT-3.5 and GPT-4 succeed at OOCR across five tasks; performance can surpass in-context learning, and GPT-4 shows stronger OOCR than GPT-3.5.The paper develops a suite of five tasks and replicates one task on Llama 3.Source: §1 · p. 2
  • Knowledge acquired without being explicitly written down may be difficult for humans to monitor, a concern connected to threat models involving deceptive models.The paper frames this as relevant to AI safety and monitoring model behavior.Source: §1 · p. 3

2 Studying inductive OOCR via finetuning

The paper defines inductive OOCR as learning latent information from training documents and using it in downstream evaluations. Its evaluation setup withholds training examples from context and tests whether models learned the latent information.

2 Studying inductive OOCR via finetuning

  • A task specifies a latent state z and training documents sampled from a training function that depends on z.The model is finetuned on those documents before evaluation.Source: §2 · p. 3
  • Evaluation queries depend on z, differ in form from the training documents, and require pretrained skills or knowledge, with no training examples supplied in context.The model can perform well only by learning z from the training data.Source: §2 · p. 3
  • Across the five tasks, evaluations can ask models to state the latent information directly; models are not finetuned on these reflection evaluations.Function addition and composition are exceptions to the general exclusion of OOCR evaluations from finetuning.Source: §2 · p. 3

3 Experiments

Across five diverse tasks, models performed above baselines; inductive OOCR generally outperformed in-context learning, and GPT-4 outperformed GPT-3.5 where compared.

3.1 Setup

  • Across five tasks, models were finetuned on indirect observations and tested on different queries without training examples in context, with performance compared against baselines.Training used varied document paraphrases; most evaluations were not included in finetuning, and the common baseline used randomly shuffled prompts of the same evaluation type.Source: §3 · p. 3, §3.1 · p. 4

3.2 Preliminary: in-context learning

  • In-context learning underperformed finetuning in every task, with little gain from 10 to 200 shots; it failed on Locations, Coins, and Parity Learning.It exceeded baselines on Functions and Mixture of Functions, but remained weaker than finetuning.Source: §3.2 · p. 5

3.3 Locations

  • GPT-3.5 named the latent city correctly 56% of the time and assigned 57% probability to the correct food with neighboring-country distractors.Far-city distance error averaged 283 km; performance dropped on out-of-distribution close and actual cities. Known cities were at least 2000 km away, and measurements were noisy.Source: §3.3 · pp. 6–7

3.4 Functions

  • Models verbalized most simple function definitions, inverted functions, and exceeded baselines by 6% on composition and 20% on addition/subtraction, but struggled with complex affine functions.The authors note that function inversion predicts x from y despite training examples showing the reverse ordering.Source: §3.4 · p. 7

3.5 Mixture of Functions

  • Without named functions or mixture hints, models learned the training distribution almost perfectly but reflected the latent set poorly in absolute terms while exceeding baseline.Models also identified the number of functions and stated the maximum or minimum of the two possible outputs.Source: §3.5 · pp. 8–9

3.6 Other tasks

  • Coins showed stochastic bias inference: 122 datapoints distinguish bias 0.7 from 0.8 at 90% confidence; Parity models assigned 80% probability to correct variable values, above baselines.For Coins, finetunes trained on bias 0.8 received consistently higher probabilities for that bias than finetunes trained on bias 0.7.Source: §3.6 · p. 9

3.7 Comparison of GPT-3.5 to GPT-4

  • GPT-4 consistently outperformed GPT-3.5 on shared OOCR tasks; Locations in-distribution error improved from ≈280 to ≈180, while Coins, Mixture of Functions, and Parity Learning showed no such improvement.The authors say the OOCR gains on those three tasks were not merely due to better in-distribution learning; possible architecture and finetuning differences leave the effect of scale uncertain.Source: Figure 4 · p. 5, §3.7 · p. 9

4 Discussion and limitations

The authors report that OOCR is prompt-sensitive and high-variance, especially for complex latent structures, and identify limits in their API fine-tuning setup and specially constructed datasets.

4 Discussion and limitations

  • OOCR is high-variance and prompt-sensitive, especially for complex structures; the authors say current models are unlikely to show it in safety-relevant contexts requiring much more complex reasoning.They nevertheless describe its emergence in simple domains as noteworthy.Source: §4 · p. 9
  • API fine-tuning prevents access to model architecture, training data, and fine-tuning algorithm, limiting careful scaling comparisons within one model family.The authors note similar results with Llama 3 fine-tuning.Source: §4 · p. 9
  • The experiments use specially created datasets and prompt formats; realistic pretraining data may be more heterogeneous and include irrelevant documents, though its greater diversity could also help OOCR.Source: §4 · p. 10

5 Related work

The paper distinguishes its focus on inferring implicit facts and testing downstream generalization from prior work on directly learned facts, latent reasoning, and other forms of generalization.

Evidence of out-of-context reasoning

  • Unlike prior work that trains directly on facts or described chatbot behaviors, this work infers implicit facts from many training documents [7] [18].Prior work also studied declarative facts affecting generations [18].Source: §5 · p. 10
  • Prior studies examine latent multi-hop reasoning, latent knowledge affecting factual learning, or learned properties in classification, but not direct verbalization plus downstream out-of-distribution evaluation [31] [16] [20].The paper connects its Locations findings to evidence of latent multi-hop reasoning without chain of thought [31].Source: §5 · p. 10

Length generalization

  • Unlike length-generalization studies focused on longer instances of the same task [5] [4] [34], this work learns latent values and evaluates on completely different tasks.Source: §5 · p. 10

Reasoning in LLMs; Knowledge Propagation

  • Unlike reasoning benchmarks [27] [26] and knowledge-editing studies [19] [22] [10] [33], this work fine-tunes latent knowledge without chain of thought and measures its propagation to downstream tasks.Knowledge editing directly trains on the fact, whereas this work does not.Source: §5 · p. 10

6 Conclusion

The paper concludes that fine-tuning on implicit information can support verbalization and downstream out-of-distribution use of latent knowledge without chain of thought. GPT-4 showed stronger OOCR than GPT-3.5, and the authors identify further directions for study.

6 Conclusion

  • Fine-tuned LLMs verbalized latent information from task-specific implicit data and used it with background knowledge and skills on downstream out-of-distribution tasks, without chain of thought.OOCR worked even where in-context learning from training examples failed.Source: §6 · p. 10
  • Future work could develop more realistic, safety-relevant tasks, study model scale, and investigate how models learn and represent latent values mechanistically.Source: §6 · p. 10

A Author contributions

  • All coauthors contributed discussions and input across the project and helped write and edit the paper.Specific task, figure, and project roles varied among coauthors.Source: A Author contributions · p. 14

B.1 Comparison of GPT-3.5 to GPT-4

  • GPT-3.5 performed similarly to GPT-4 on the training task but better on OOCR evaluations in Coins, Mixture of Functions, and Parity Learning.Source: Figure 10 · p. 14, §B.1 · p. 14

B.2 In-Context Learning experiments

The appendix compares in-context learning with fine-tuning on OOCR evaluations and provides expanded Locations results for GPT-3.5. In-context learning did not exceed its baseline across the evaluations described.

B.2 In-Context Learning experiments

  • The appendix adds a comparison of in-context learning and inductive OOCR on both in-distribution task performance and OOCR evaluations.Source: Figure 11 · p. 14, §B.2 · p. 14

C.1.1 In-Context Results

  • In-context learning did not outperform its baseline on any evaluation; the comparison includes training-task performance and OOCR evaluations.For multiple-choice evaluations, the baseline is untrained-model performance.Source: Figure 12 · p. 15, §C.1.1 · p. 15

C.1.2 GPT-3.5 Full Results

  • The expanded GPT-3.5 Locations results add multiple-choice evaluations with varied distractors and a Reversal evaluation.Except for Free-form, the evaluations use multiple-choice format, with distractors specified by the column label.Source: §C.1.2 · p. 15, Figure 13 · p. 16

C.1.3 GPT-4 Results

Fine-tuned GPT-4 outperforms GPT-3.5 on unknown-city OOCR and out-of-distribution distance prediction, while in-context learning remains below fine-tuning. The appendix describes the location-task data and evaluation design, including limits imposed by city familiarity.

C.1.3 GPT-4 Results

  • GPT-4 models outperform GPT-3.5 on unknown-city OOCR and out-of-distribution distance prediction, while in-context learning remains worse than fine-tuning.The out-of-distribution distance test uses cities less than 2,000 km from unknown places.Source: Figure 14 · p. 16, §C.1.3 · p. 16

C.2 Training Dataset

  • Training examples pair unknown places with known-city distances or directions; distance responses include noise, unit variation, and rounding.The dataset contains 25,225 examples, with 500 unique city comparisons per unknown place and five prompt phrasings per tuple.Source: §C.2 · pp. 16–17

C.3 Evaluation Dataset

  • Evaluation tests distance prediction on unseen cities and OOCR through questions about each unknown place’s country, city, and food.Multiple-choice and free-form formats help account for prior answer preferences and refusal behavior.Source: §C.3 · p. 17

C.3.1 Country Query; C.3.2 City Query

  • Country and city queries vary distractors, including other unknown places, populous options, nearby-country choices, and reversal questions.The closest-country design avoids a query that is too easy because of a strong preference for the central option.Source: §C.3.1 · pp. 18–19, §C.3.2 · pp. 19–20

C.3.3 Food Query

  • Food queries use multiple-choice options drawn from other unknown-place countries or nearby countries, with three sampled choice sets.Foods were generated with GPT-4 and retained only when GPT-3.5 could identify their country.Source: §C.3.3 · p. 21

C.4.1 Choice of cities

  • The city selection favors well-known places because pretraining priors can lead models toward the wrong city, especially for less prominent locations.The authors report distinguishing populous cities such as London and Paris, but difficulty distinguishing Paris from a small French city.Source: §C.4.1 · p. 22

C.4.2 Measuring Performance; C.4.3 Baselines

  • Distance performance is summarized by median absolute error, while OOCR scoring uses response probabilities and excludes refusals from free-form estimates.Multiple-choice baselines test untrained preferences; the in-context baseline uses the best result among K = 10, 100, and 200 examples.Source: §C.4.2 · p. 22, §C.4.3 · pp. 22–23

C.4.4 Fine-tuning Details; D.1 Overview

  • The location experiments use ten GPT-3.5 and five GPT-4 fine-tuning runs, while the Coins task requires aggregating stochastic flip outcomes.Coins training examples each contain one flip; distinguishing biases 0.7 and 0.8 with at least 90% confidence requires at least 122 independent flips.Source: §C.4.4 · p. 23, §D.1 · p. 24

D.2 Results

On the Coins task, GPT-4 performs well across downstream evaluations, while GPT-3.5 is above baseline on directional-bias tasks but fails to distinguish biased from fair coins. Reflection performance is above baseline but low, and models show a preference for reporting 0.8 bias.

D.2 Results

  • On reflection tasks, all models favor the 0.8 answer, and performance is above baseline but low, with no GPT-3.5–GPT-4 difference.Models trained on 0.8 bias show a stronger preference for that answer, contributing to positive overall performance.Source: §D.2 · p. 24

D.3 Training dataset

  • Coins training encodes stochastic single-flip outcomes across four coins, with outcome frequencies determining each coin’s bias.Runs use either 0.7- or 0.8-biased coins, alongside two fair coins and a coin biased toward tails.Source: §D.3 · p. 24, Table 3 · p. 26

D.2 Results

  • GPT-4 performs well across non-reflection Coins evaluations, while GPT-3.5 exceeds baseline on most but fails to distinguish biased from fair coins.Both model groups are above baseline but perform poorly on reflection evaluations.Source: §D.2 · p. 24, Figure 15 · p. 25

D.4.1 Reflection; D.4.2 Other

  • Reflection probes exact bias values, while other evaluations test bias direction or existence and its consequences through betting and coin selection.The evaluation set includes probability estimation, biased-versus-fair judgments, reversal, and strategic betting tasks.Source: §D.4.1 · pp. 25–26, §D.4.2 · pp. 27–28

D.5 Training performance

Models learn stronger coin biases than the training probabilities, complicating interpretation of low reflection scores; recalculating reflection against learned probabilities yields inconclusive results. The appendix estimates the coin samples needed to distinguish two biases at specified error rates.

D.5 Training performance

  • Because Coins training tasks are stochastic, models cannot achieve perfect training performance; although matching ground-truth probabilities minimizes mean cross-entropy, models instead learn a much stronger bias.On Reflection tasks against learned bias, multiple-choice performance does not change; free-form performance decreases for GPT-3.5 and increases for GPT-4.Source: §D.5 · p. 29, Figure 17 · p. 30
  • The authors do not know why models learn stronger biases and offer RLHF mode collapse and unknown API fine-tuning logic as possible explanations.Neither explanation is established as the cause.Source: §D.5 · p. 30
  • Recomputing reflection scores using learned rather than ground-truth biases leaves results inconclusive about whether low scores reflect incorrect learning or weak reflection.The recalculation uses coin probabilities 0.03, 0.17, 0.5, 0.82, and 0.97.Source: §D.5 · p. 30

D.6 Required number of training examples for Coins

  • For biases 0.7 and 0.8, distinguishing the coin with 90% confidence requires at least 122 flips, and 200 flips for 95%.These sample counts are derived under an equal prior over the two possible biases.Source: §D.6 · pp. 30–31

E.1.1 Inductive OOCR results

The appendix presents additional Functions-task results across overall evaluations, individual functions, regression inputs, and function composition or arithmetic operations.

E.1.1 Inductive OOCR results

  • Figure 18 reports overall Functions performance across the evaluations.The passage directs readers to Appendix E.6 for evaluation and baseline descriptions.Source: Figure 18 · p. 32, §E.1.1 · p. 32
  • Figure 19 reports free-form reflection and function inversion results for each of the 19 functions.The results separate regression-only functions from functions trained with augmentations.Source: Figure 19 · p. 32, §E.1.1 · p. 32
  • Figure 20 shows regression-task performance for in-distribution and out-of-distribution inputs.Source: §E.1.1 · p. 32, Figure 20 · p. 33
  • Figure 21 reports function-specific composition and addition/subtraction results, including held-out tests for augmented functions.Regression-only functions were not trained on these tasks; integer addition and subtraction results are omitted.Source: §E.1.1 · p. 32, Figure 21 · p. 33

E.1.2 In-context learning results

This section reports in-context learning evaluations, including function-output-type choices and a GPT-3.5/GPT-4 comparison for Mixture of Functions; it also summarizes performance relative to baseline and OOCR and provides results for large constants and training and evaluation procedures.

E.1.2 In-context learning results

  • In-context learning exceeds baseline on every evaluation, but overall trails OOCR; more shots scarcely change OOCR performance, though composition can work better with ICL.Training-task performance rises slightly from 10 to 100 shots, then levels off.Source: Figure 22 · p. 34, §E.1.2 · p. 34

E.2 Results for addition/subtraction with large constants

  • The large-constant experiment shows Python definitions and function inversions even for arbitrary constants such as x + 883.It reused the main Functions setup with a different function list and omitted in-context learning to save costs.Source: Figure 24 · p. 35, §E.2 · p. 35

E.3 Additional discussion

  • x + 5 showed no free-form reflection OOCR, while x − 1 reached around 65%; proposed explanations include exact-constant errors, variable-name effects, and experimental noise.Responses for x + 5 often gave nearby constants, and variable names beginning with “py” sometimes triggered misinterpretations.Source: §E.3 · p. 37

E.4 Finetuning hyperparameters; E.5 Training task details

  • Training sampled regression and augmentation prompts, using function-output examples and augmented-function combinations.Finetuning used 96,000 datapoints for one epoch of 1,500 steps, with batch size 64; the large-constant task used inputs from −1999 to 1998.Source: §E.5 · pp. 37, 39, §E.4 · p. 37

E.6 Evaluation details; E.6.1 Training evaluations

  • Evaluation estimated target probabilities from token log-probabilities or sampled outputs, and generally used 1,000 prompts per evaluation.Free-form definition used 2,000 prompts; training evaluations included held-out in-distribution and out-of-distribution prompts, with baselines based on random alternative targets.Source: §E.6 · p. 40, §E.6.1 · p. 40

E.6.2 Combination evaluations

  • Add/Subtract and Composition tested combining regression-trained functions on augmentation prompts, with evaluation inputs restricted to −15 through 14.This was the paper’s only OOCR evaluation trained on examples of the OOCR task, included because the task was otherwise too hard; Boolean-output functions were excluded.Source: §E.6.2 · pp. 40–41

E.6.3 Reflection evaluations

  • Reflection evaluations asked models to identify a function from its variable name, using free-form lambda generation or multiple-choice formats.Free-form outputs were checked by running the code, while reflection baselines measured the tendency to select functions regardless of the specific variable.Source: §E.6.3 · p. 41

E.6.4 Other evaluations

  • Other evaluations tested function inversion, matching a function to its variable, and determining a function’s output type.In inversion, any input in the output’s preimage counted as correct.Source: §E.6.4 · pp. 42–43
  • The output-type evaluation samples functions with three output types in equal proportions and asks the model to choose the correct type.The example prompt offers bool, float, and int as options and requests a single uppercase letter.Source: §E.6.4 · p. 43

F.1.1 Inductive OOCR results

  • Figure 27 restates GPT-3.5 results and compares them with GPT-4 on Mixture of Functions, covering regression predictions and Reflection and Other evaluations.Output 1–3 correspond to predictions with 0, 1, and 2 in-context examples, respectively.Source: Figure 27 · p. 44, §F.1.1 · p. 44

F.1.2 In-context learning results

In the Mixture of Functions in-context evaluation, performance exceeded baseline in most evaluations but worsened with more shots and remained below finetuning. The accompanying methods describe the function and parity training data and evaluation protocols.

F.1.2 In-context learning results

  • In-context learning beat baseline in most Mixture of Functions evaluations, worsened with more shots, and performed worse overall than finetuning.Figure 28 reports the detailed in-context learning results.Source: §F.1.2 · p. 44, Figure 28 · p. 45

F.2 Finetuning hyperparameters

  • GPT-3.5 finetunes used two-function subsets and 6,000 datapoints for 1,500 steps; GPT-4 used one finetune per subset.Both used batch size 4 for one epoch; the GPT-3.5 and GPT-4 model versions differed from other tasks for API-limit reasons.Source: §F.2 · p. 45

F.3 Training task details

  • Training prompts sampled one of two functions and showed three input-output examples, with varied instructions and formats that avoided revealing the candidate function set.Inputs were sampled from −99 to 98; half the examples included x and y labels and half showed integers alone.Source: §F.3 · pp. 45–46

F.4 Evaluation details

  • Evaluations sampled five responses for multi-token targets and assessed training outputs, including distribution predictions before and after the model could infer the sampled function.The training-task metric was one minus average total variation distance, equivalent to target probability for deterministic targets.Source: §F.4 · p. 46, §F.4.1 · p. 46

F.4.2 Reflection

  • Reflection tests asked models to identify the two possible functions from multiple-choice options, scoring correct only when both functions were selected and no others.Prompts included redacted training examples.Source: §F.4.2 · p. 47

F.4.3 Other

  • Other evaluations probed possible outputs and the number of candidate functions; the untrained GPT-3.5 model assigned roughly 20% to the correct function-count answer, while GPT-4 gave substantially less.The number-of-functions evaluation used a forced choice among letters A–E.Source: §F.4.3 · pp. 48–49

G Parity Learning task details

  • The parity task encoded eight binary variables, with four set to 1 and four to 0, and trained models to answer parity queries over sampled tuples.Queries used tuples of four, five, or six variables, sampled with replacement.Source: G Parity Learning task details · p. 50

G.1 Training Dataset; G.2.1 Training task evaluation

  • The parity dataset contained 32,000 syntactically augmented prompts; evaluation included held-out prompts and parity queries with out-of-distribution numbers of variables.The length evaluation tested variable counts from 1 to 11, excluding 4, 5, and 6.Source: §G.1 · p. 50, §G.2.1 · p. 51

G.2.2 OOCR evaluation

The parity OOCR evaluations test whether models can use learned variable values across altered query formats and contexts. The section also explains response scoring, comparison baselines, and model training setups.

G.2.2 OOCR evaluation

  • OOCR probes tested direct variable recall, string formatting, division, conditional word selection, and equality questions using learned variables.The direct-print task included both Python and natural-language prompts.Source: §G.2.2 · pp. 52–54
  • Additional probes reversed recall by asking for variable names from values, or mixed learned variables with in-context integers or explicitly defined variables.Mixed-context tests used one, two, or three additional variables.Source: §G.2.2 · pp. 52, 54

G.3.1 Measuring Performance

  • For each evaluation prompt, target probability was estimated from ten temperature-1 samples; invalid answer formats could depress finetuned scores, so overview plots normalized valid-response probabilities.The authors note that finetuned models often produced invalid answers, likely from overfitting to the narrow training distribution.Source: §G.3.1 · p. 54

G.3.2 Baselines

  • Finetuned models were compared with overall target-probability and in-context-learning baselines, with the best result reported across 10, 100, and 200 examples.The overall-target baseline ignores the prompt; in-context examples were prepended to each evaluation prompt.Source: §G.3.2 · p. 55

G.3.3 Finetuning Details

  • GPT and Llama parity runs used different training setups: GPT models trained on 32,000 datapoints for one epoch, while Llama runs used 384,000 datapoints.GPT-3.5 and Llama runs used ten repetitions; GPT-4 used five, and Llama models were finetuned with LoRA.Source: §G.3.3 · p. 55

G.4 Results on GPT

On Parity Learning, finetuned GPT models outperform their baseline, while GPT-3.5 in-context performance is weaker; Llama 3 models also beat baseline, with results varying by evaluation type.

G.4 Results on GPT

  • GPT-3.5 assigns an average 80% probability to true variable values in natural-language and Python queries, and GPT-4 performs better than GPT-3.5.Finetuned performance is above baseline; GPT-3.5 in-context performance is close to 60%.Source: §G.4 · p. 55, Figure 29 · p. 56

G.5 Results on Llama3

  • Both Llama 3 8B and 70B perform better than baseline, but 8B leads on print reflection evaluations while 70B leads on string and control evaluations.Source: §G.5 · p. 55, Figure 30 · p. 56

G.6 In-Context Results

  • GPT-3.5 in-context learning performs worse than its baseline on almost all evaluations, including in-distribution evaluations.Source: §G.6 · p. 55, Figure 31 · p. 57

Related papers

Loading 2406.14546v3…