Automated Capability Discovery via Foundation Model Self-Exploration

1 Introduction

• Human-created benchmarks are labor-intensive and bounded by predefined categories, and may miss surprising behaviors, unknown unknowns, or performance differences as models saturate them.Continually creating new test suites is expensive and not model-specific.Source: §1 · pp. 1–2
• ACD assigns a foundation model as a scientist to propose open-ended tasks for a subject model, then evaluates the subject’s performance automatically.The subject may be the scientist model itself or a different model.Source: §1 · p. 2
• ACD generated thousands of tasks across several foundation models, with task families spanning 25 high-level categories and human surveys supporting the validity of tasks and agreement with automated scoring.The categories include puzzle-solving, code generation, scientific reasoning, creative writing, and legal interpretation.Source: §1 · pp. 2–3
• ACD’s automated discovery and self-assessment could help identify beneficial and concerning behaviors before deployment, while potentially supporting future model self-improvement.The paper presents these as potential future applications.Source: §1 · p. 2

2.1 Open-ended Discovery Algorithms

• Open-ended algorithms seek to generate novel and diverse artifacts continuously, rather than optimize a fixed objective.Foundation models can contribute an intrinsic notion of interestingness to artifact generation.Source: §2.1 · p. 3
• A typical algorithm samples an artifact conditioned on a subset of an archive, evaluates its novelty, and adds it if sufficiently different from existing artifacts.The context subset is typically limited for computational feasibility.Source: §2.1 · p. 3
• ACD adapts this archive-based approach by treating each discovered task that the model succeeds or fails on as an artifact.Source: §2.1 · p. 3

2.1 Open-Ended Discovery with Foundation Models

• Open-ended discovery methods use foundation models’ generative abilities and notion of interestingness to propose novel artifacts, including robot morphologies, environments, loss functions, agent designs, and scientific ideas.Examples include ELM, OMNI-EPIC, DiscoPOP, ADAS, and The AI Scientist.Source: §3 · p. 3

2.2 Automated Evaluation of Foundation Models

• Automated evaluation work includes safety red-teaming and domain-focused generation, but these approaches do not broadly generate entirely new task families for capability discovery.Some generated tasks focus on restricted domains, while benchmark augmentation typically modifies existing benchmarks or task structures.Source: §3 · pp. 3–4
• ACD emphasizes broad, de novo discovery of task families; direct quantitative comparison is challenging because established baselines for wide-ranging exploration are absent.Its outputs may complement focused evaluation efforts by supplying novel task types or identified failure modes.Source: §3 · p. 4

4 Automated Capability Discovery

• ACD designates a foundation model as scientist to propose tasks and evaluate a subject model, which may be the same model or a different one.The framework aims to automate much of the process of revealing model capabilities through iterative refinement.Source: §4 · p. 4

4.1 Definition of Task Families

• A task family groups tasks probing a capability and includes metadata naming the family, describing it, and specifying the capability measured.Each family contains task instances, subject instructions, and a scoring mechanism, using programmatic checks or a GPT-4o-based judge for open-ended evaluation.Source: §4.1 · p. 4

4.2 Generating Tasks

• ACD maintains an archive seeded with trivial tasks; the scientist samples prior tasks, proposes a family, and checks whether the subject can solve it to adapt difficulty.Source: §4.2 · pp. 4–5
• Proposals are filtered for novelty using embedding-based nearest neighbors, then tested by the subject with chain-of-thought and n-shot scoring; consistent successes or failures are logged as capabilities or failure modes.Source: §4.2 · p. 5
• Repeated iterations and HDBSCAN clustering group similar tasks, allowing consistent performance across instances to support higher-level behavioral characterizations.Source: §4.2 · p. 5

5.1 Case Study and Human Evaluation on GPT-4o

• 1,330 tasks from 5,000 generations formed 25 clusters spanning capabilities from puzzle-solving and code to creative writing and legal interpretation.The clusters represent different capability areas or failure patterns.Source: §5.1 · p. 5
• ACD exposed both granular failures and unexpected successes, including arithmetic and pattern errors alongside solving a novel 17-clue logic puzzle.The authors report a cost of approximately $0.1 per task created.Source: §5.1 · p. 5
• 92.2% of tasks were judged clear and valid, and automated scoring reached an F1 of 0.86 against human judgments, with agreement lower on very difficult tasks.The automated judge showed a slight positive bias.Source: Figure 3 · p. 6, §5.1 · p. 6
• Around 20% of newly proposed tasks remained interestingly new after 5,000 generations, while repeated seeds yielded consistent final task collections.The findings suggest ACD had not fully uncovered GPT-4o’s capabilities.Source: §5.1 · p. 6

5.2 Varying the Subject Model and Cross-Model Analysis

• On GPT-4o-discovered tasks, Llama3-8B’s performance was nearly a subset of GPT-4o’s, with a narrower gap on imaginative or open-ended generation.Llama3-8B struggled more on multi-step reasoning and structured workflows; because tasks were tailored to GPT-4o, some may have been too difficult for Llama3-8B.Source: Figure 4 · p. 6, §5.2 · p. 7
• When GPT-4o generated tasks for Llama3-8B, ACD found 1,395 new tasks and failures including a trivial matrix rotation and a logic puzzle loop.The authors describe some failures as new discoveries for the GPT-4o scientist, given its October 2023 knowledge cutoff and Llama3-8B’s July 2024 release.Source: §5.2 · p. 7

5.3 Varying the Scientist Model

• With GPT-4o as subject, Claude Sonnet 3.5 discovered 2,873 tasks across 46 clusters and proposed more abstract, interdisciplinary, and creative probes than GPT-4o.Sonnet’s task set broadly covered GPT-4o’s regions while adding open-ended or creatively oriented tasks not covered by GPT-4o’s proposals.Source: §5.3 · p. 7, Figure 5 · p. 7, Figure 6 · p. 8

5.4 Report Generation

• ACD compiles tasks, responses, and evaluations into a Capability Report with notable examples, cluster explanations, and an overall summary of capabilities and mistakes.The report offers an interpretable reference and reduces manual curation effort, though manual review remains useful for selecting illustrative examples.Source: §5.4 · pp. 8–9

6 Safety Considerations

• Containerized execution restricts network and host-filesystem access, while static analysis found no attempted internet or filesystem access by ACD.The authors say these measures substantially reduce the likelihood of dangerous code deployment.Source: §6 · p. 9
• ACD could help safety teams identify unexpected capabilities or risks, such as guardrail circumvention or flawed legal interpretations, for deeper investigation before deployment.The paper presents this as a contribution to more comprehensive pre-deployment assessment, not a standalone safety solution.Source: §6 · p. 9
• The authors identify finding capabilities or risks entirely unanticipated by developers as a target for future work.Source: §6 · p. 9

• ACD autonomously discovers and evaluates foundation-model capabilities, reducing the need for manual task design.Source: §7 · p. 9
• Human evaluation of GPT-4o tasks finds most generated tasks coherent and self-assessment reasonably aligned with human judgments.Source: §7 · p. 9
• Experiments focus on single-turn, text-based tasks; future extensions could target agentic or multimodal tasks.Source: §7 · p. 9
• Generated tasks could also provide new challenges for models to solve themselves, potentially facilitating self-improvement.Source: §7 · p. 9

A Task Code

• Task-family examples illustrate implementations for both simple and more open-ended tasks.Source: A Task Code · p. 16

A.1 Example Task Family Code

• The Hello World example follows a simplified METR Task Standard and scores whether the submission exactly matches the required message.Source: §A.1 · p. 16

A.2 Evaluating Free-Form Responses Using an LLM Judge

• For creative or reasoning tasks that resist simple programmatic checks, an LLM judge evaluates free-form responses against criteria such as length, structure, and prompt adherence.Source: §A.2 · pp. 16–17

B ACD Prompts

• The appendix provides the complete set of prompts used by the scientist model in ACD.Source: B ACD Prompts · p. 17

B.1 Task Creation Prompts

• Task-creation prompts request novel, diverse task families and specify fields including task description, measured capability, estimated difficulty, and implemented code.Source: §B.1 · pp. 17–18
• The scientist receives prior tasks for context and uses reflection prompts to check implementation correctness, task clarity, and evaluation feasibility before marking a task done.Source: §B.1 · p. 19

B.2 Evaluation Prompts

• Subject evaluation prompts support both direct zero-shot answers and chain-of-thought reasoning before the final answer.Source: §B.2 · p. 19

B.3 Task Embedding Prompt

• Task embeddings use text-embedding-3-small and include the task name, description, measured capability, estimated difficulty, example instruction, and success status.Source: §B.3 · p. 20, Task Embedding Prompt · p. 20

B.4 Novelty Assessment Prompts

• Novelty assessment compares a proposed task family with its closest existing neighbors and considers subject-area novelty, difficulty, measured capabilities, and task success.Source: §B.4 · p. 20

B.5 LLM Judge Prompts

• The LLM-judge prompt evaluates whether a response succeeds using factors such as relevance, accuracy, creativity, and optional additional criteria.Source: §B.5 · pp. 20–21

C Hyperparameters

• The appendix lists experimental sampling and algorithm parameters, notes that settings are consistent across evaluated models, and specifies additional t-SNE and HDBSCAN parameters for clustering.Source: C Hyperparameters · p. 21, Table 2 · p. 21

• $450 total, or approximately 10 cents per generation, for GPT-4o scientist-on-GPT-4o-subject experiments.Source: §C.1 · p. 21
• GPT-4o scientist-on-Llama3-8B experiments cost slightly less because the subject model was cheaper.Source: §C.1 · p. 21
• Sonnet 3.5–GPT-4o experiments were approximately 50% more expensive.Source: §C.1 · p. 21

D.1 Additional Analysis

• After thousands of generations, ACD still discovers novel tasks, and three 500-generation GPT-4o runs show broadly consistent capability distributions across seeds.The authors also report that tasks span each difficulty category and that human-estimated success drops steeply as difficulty rises, unlike FM-judge success.Source: §D.1 · p. 23, Figure 8 · p. 23, Figure 9 · p. 23
• ACD discovers tasks across difficulty categories, while human-estimated success drops steeply with difficulty and FM-judge success varies little.The authors interpret coverage across difficulty categories as suggesting that ACD can adapt task difficulty to the subject model’s capability.Source: §D.1 · pp. 23–24, Figure 10 · p. 24

D.2 Additional Visualizations for Llama3-8B as Subject

• When GPT-4o probes Llama3-8B, the tasks overlap partly with self-evaluation tasks, but Llama3-8B fails more often on multi-step logic and advanced reasoning.The authors conclude that ACD can adaptively explore areas of potential failure in the subject model.Source: §D.2 · p. 24, Figure 11 · p. 24

E Examples of Discovered Tasks

• Discovered tasks range from basic transformations to cryptography, linguistics, and complex puzzles, offering examples of capabilities and potential blind spots.One selected success reconstructs a square pyramid from 2D projections.Source: E Examples of Discovered Tasks · pp. 24–25, §E.3.2 · p. 30

E.1 Listing of Discovered Clusters

• Tables 4–6 list the primary clusters for three scientist–subject configurations, sorted by cluster size and accompanied by subject-model FM-judge success rates.The tables cover GPT-4o on GPT-4o, GPT-4o on Llama3-8B, and Sonnet 3.5 on GPT-4o.Source: §E.1 · p. 25, Table 4 · p. 25, Table 5 · p. 26

E.2 Manual Selection of Surprising Tasks

• Because crowd judgments and model predictions did not reliably isolate genuinely surprising tasks, the authors manually curated the examples.About 37% of tasks were marked surprising by crowd participants; the authors say that labeling was too broad for their goal.Source: §E.2 · p. 28

E.3.1 Selected Failures

• In a selected arithmetic example, GPT-4o returns 10 after the prompt’s operations on 5, illustrating a failure on a sequential calculation.The specified operations are add 3, multiply by 2, then subtract 4.Source: §E.3.1 · p. 28

E.3.2 Selected Successes

• A selected success describes the object implied by the projections as a square pyramid, including its four-unit square base and three-unit height.The authors present selected examples of surprising successes and failures when GPT-4o serves as both scientist and subject.Source: §E.3 · p. 28, §E.3.2 · p. 30

E.4.1 Selected Failures

• Llama3-8B reported a $382.00 monthly payment for the $20,000, 5-year loan at 5% annual interest.The response used a monthly rate of 0.004167 and 60 payments before giving that result.Source: §E.4.1 · pp. 31–32
• In the shopping problem, Llama3-8B’s solution reaches $337 after multiplying 96 cookies by the per-box price and applying a $14 cracker discount.The prompt gives a price per box of cookies and a discount of $2 for every two cracker boxes; the response instead treats the cookie price as per cookie and each cracker pair as costing $2.Source: §E.4.1 · p. 32
• The family-tree response lists John and Mary followed by Jack, Lily, and Sarah, rather than representing the described generations and relationships.The task description names Alice and Bob as John and Mary’s children, with Emma, Jack, and Lily in the next generation.Source: §E.4.1 · pp. 32–33
• In the logic puzzle, the response incorrectly places the blue house fifth, then repeats contradictory claims about whether Pall Mall is in the first or second house.It cycles between the Norwegian in the first house and the Swede in the second without resolving the deduction.Source: §E.4.1 · pp. 33–34
• The cryptarithm response gives impossible carry deductions, including D + 9 = -1, and repeats those contradictions.It also claims an intermediate carry must be 9.Source: §E.4.1 · p. 35

E.5.1 Selected Failures

• GPT-4o’s substitution cipher uses animal words rather than the required color words for its key and encoded message.The task specified a cipher themed around colors, but the response assigns words such as dog, cat, and fish to letters.Source: §E.5.1 · pp. 35–36

E.5.2 Selected Successes

• In the subsection labeled a success, GPT-4o responds to a Lojban conditional-sentence task with an Ithkuil description and a sentence translated as “A wave of bittersweet nostalgia washed over him.”Its explanation also analyzes the sentence as Ithkuil rather than Lojban.Source: §E.5.2 · p. 36
• The ecosystem response identifies equilibria for a four-component model and predicts that added predators or reduced crystal light could destabilize populations or cause collapse.It says the all-species equilibrium is stable if the Jacobian has eigenvalues with negative real parts.Source: §E.5.2 · p. 38
• The language response proposes communicating through particle quantum states, with redundant encoding across electrons and photons to withstand electromagnetic noise.Changes in spin states are described as encoding linguistic units, while redundancy and error-correcting codes address environmental fluctuations.Source: §E.5.2 · pp. 38–39

2. Specific Aspects

• The response represents emotions through sound-frequency and amplitude patterns, while describing abstract ideas through quantum metaphors such as resonance and equilibrium.It associates higher frequencies and amplitudes with excitement or urgency, and lower, softer signals with calmness or sadness.Source: §2 · p. 39

3. Examples of Expressing Concepts

• Time is represented by quantum-state transitions, quantity by simultaneous state changes, and relationships by arrangements such as entangled or nearby states.The response associates entangled states with familial bonds, nearby unentangled states with social relations, and particle positions with spatial relations.Source: §3 · p. 39

4. Sample Communication Unit

• A sample utterance combines spin-state transitions, a photon shift, and frequency modulation to mean, “We are moving towards an exciting event.”The response assigns temporal, informational, and emotional functions to those elements, respectively.Source: §4 · p. 39

5. Challenges for Humans

• The response says humans would struggle to produce or understand the language because they cannot directly perceive quantum states and would need specialized equipment.It also cites the system’s complexity and quantum metaphors as barriers, while suggesting advanced technology might enable translation interfaces.Source: §5 · pp. 39–40

F Example Failures of the Automated Judge

• The automated judge sometimes marks successful tasks as failures, including when response parsing is non-robust.Examples include a frustum-volume response and an algebraic-equation response.Source: F Example Failures of the Automated… · p. 40

G Human Surveying Details

• Human evaluations used anonymous task-labeling surveys with CloudResearch participants experienced with large language models.Participants had up to two minutes per task; the survey asked about task fit, difficulty, correctness, and surprise.Source: G Human Surveying Details · pp. 40–41

H Report Generation

• ACD can automatically generate structured reports summarizing discovered capabilities, consistent successes and failures, and key insights.The workflow aggregates and summarizes clusters and tasks.Source: H Report Generation · p. 42

H.1 Task Cluster Labeling

• GPT-4o labels task clusters after t-SNE and HDBSCAN grouping, using specific natural-language labels of at most 10 words and a stated overall capability.The labels are used to summarize the discovered tasks in the final analysis.Source: §H.1 · p. 42, [INSTRUCTION] · p. 42

H.2.1 Cluster Analysis Prompts

• Cluster-analysis prompts ask the model to examine task responses, performance statistics, and surprising examples, then report capabilities, limitations, patterns, and example-specific insights.The requested output includes an overall analysis and a separate analysis for each provided example.Source: §H.2.1 · p. 43

H.2.2 Example Selection Prompts

• Example-selection prompts ask the model to choose up to three surprising successes and failures, explaining which examples are unexpected from an expert researcher’s perspective.The model is asked to reason briefly about every example, including those it does not select.Source: §H.2.2 · p. 44

H.2.3 Overall Summary Prompts

• Overall-summary prompts request a synthesis across clusters, including an abstract, capability and limitation summaries, surprising findings, and interpretation of numerical data.The prompt asks the report to reference cluster examples and discuss patterns, success rates, and performance statistics.Source: §H.2 · p. 42, §H.2.3 · p. 45

H.3 Generated Report Example

• The generated report example presents a broad assessment of GPT-4’s capabilities across procedural tasks, scientific reasoning, legal analysis, and other clusters.The appendix identifies the report as an ACD example for GPT-4o serving as both scientist and subject, while its abstract describes GPT-4 as both.Source: §H.3 · p. 46, ABSTRACT · p. 47

1 Overview

• The sample overview describes strengths in structured procedural tasks, scientific communication, and legal reasoning, including origami instructions and legal-text interpretation.It also reports proficiency in scientific reasoning while noting difficulty operationalizing experimental designs for abstract phenomena.Source: §1 · p. 48
• The sample overview reports difficulties with dynamic and abstract problem-solving, including complex pathfinding, chess strategy, and advanced mathematical reasoning.It characterizes these as limitations in spatial reasoning and domain-specific adaptation, while noting high success rates in scientific reasoning and historical analysis clusters.Source: §1 · p. 48
• Figure 2 presents success rates on each cluster of tasks.Source: §1 · p. 49
• The model struggled with dynamic strategy and advanced mathematical reasoning, and showed inconsistent logical performance between abstract reasoning and basic proof construction.The overview also reports strengths in scientific communication and legal reasoning.Source: §1 · p. 49

2 Detailed Task Analysis

• 92.94% success across 34 procedural-generation and troubleshooting tasks included 95.00% on moderate and 91.50% on hard tasks.The analysis characterizes these tasks as testing procedural understanding, technical communication, and instructional clarity.Source: §2 · p. 50
• The model showed strong sequencing and spatial reasoning on origami instructions, including 100.0% success on an example requiring it to describe the resulting creased shape.The analysis notes that less structured or more abstract procedural tasks may remain challenging.Source: §2 · p. 50