PaperlayerView exact revision on arXiv

Source-linked AI summary

Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist

Akshay K. Jagadish, Younes Strittmatter, Nori Jacoby, George Kachergis, Eric Schulz, Nathaniel Daw, Suyog H. Chandramouli, Thomas L. Griffiths

arXiv:2606.26448v1q-bio.NCcs.AI

TL;DR

Theory-building remains a largely manual step in cognitive science despite advances in data collection, modeling, and experimental design. AUTOCOG uses LLM agents to run a closed discovery loop, recovering strategies in simulation and producing human-behavior theories that outperform seed theories and generalize to held-out studies.

  • Problem

    Data collection, model discovery, and experimental design have advanced, but using failures of existing models to build better psychological theories remains largely manual.

    Source: Introduction · p. 1

  • Method

    AUTOCOG uses LLM agents to propose executable theories, design discriminating experiments, collect behavioral data, compare predictions with observations, and revise theories.

    Source: Introduction · p. 2

  • Results

    AUTOCOG recovered known and unconventional strategies in simulations, while human-run theories outperformed seed theories and generalized to held-out studies.

    Source: Discussion · p. 11

  • Takeaways & Limitations

    Machine-readable traces of experiments, data, verdicts, and theories make discovery runs auditable and allow researchers or agents to resume a search from its frontier.

    Source: Discussion · p. 11

    Search is bounded by researcher-defined experiment and model spaces, while parsimony, unification, and interpretability are not explicit objectives.

    Source: Discussion · p. 11

Abstract

from arXiv · show

Across the sciences, autonomous systems are increasingly being used in closed-loop discovery, proposing new theories and designing and running experiments to test them. This approach is yet to be applied in the field of cognitive science, where the central bottleneck is theory-building: the creative step of turning the accumulated failures of existing models into better ones. Theory generation has remained manual even as data collection, modeling, and experiment design have been automated. We present the Automated Cognitive Scientist (AutoCog), a fully autonomous agentic-AI system that closes this loop. Large-language-model agents advocate competing theories, each expressed as an executable cognitive model, design experiments that best discriminate them, collect behavioral data from participants recruited online, score theories against collected data based on their generative performance, diagnose why they fail, and synthesize a better successor. Repeating this cycle allows them to search the space of theories, models, and experiments. In the domain of decision-making, AutoCog recovered known decision-making strategies from simulated behavior, including unconventional ones, showing that its discoveries are ultimately driven by the data rather than strictly bound by the priors of the underlying language models. When run with human participants, it produced theories that outperformed the established theories it was seeded with and generalized to held-out studies in two different experimental settings. It also surfaced a novel theory of multi-cue decision-making in which choices show diminishing sensitivity to feature values. The distinctive predictions of this theory were confirmed in a preregistered study with new participants. AutoCog demonstrates how an automated discovery system can be used to turn cognitive theory-building into an explicit, executable, and cumulative science.

Introduction

Psychology has automated data collection, modeling, and experiment design, but theory-building remains a conceptual bottleneck: turning existing models’ failures into better ones. AUTOCOG addresses this gap with a closed-loop system for discovering psychological theories from human behavior.

Introduction

  • Psychology can automate much of the discovery cycle because online human experiments and code-based analysis complement advances in data collection, model discovery, and experimental design.These capabilities had accelerated, while theory generation remained manual.Source: Introduction · pp. 1–2
  • The central challenge is conceptual: researchers must use accumulated empirical failures of existing models to imagine better psychological theories.Psychological theories must make opaque processes mechanistically meaningful rather than being directly compiled or solved.Source: Introduction · pp. 1–2
  • The decision-making task varied expert ratings across binary and cardinal cue formats, providing two settings for evaluating AUTOCOG.Binary ratings were 0 or 1; cardinal ratings ranged from 0 to rmax.Source: Figure 1 · p. 2, Introduction · p. 2
  • AUTOCOG uses LLM agents to propose executable theories, design discriminating experiments, collect human behavior, evaluate predictions, and synthesize successor theories.In decision-making, its human-run theories outperformed seed theories and generalized to held-out studies; it also surfaced a diminishing-sensitivity account confirmed in a preregistered study.Source: Introduction · p. 2

Automated cognitive scientist (AUTOCOG)

AUTOCOG operationalizes theory-building as a repeated cycle in which competing accounts guide experiments, data collection, comparison, and revision. Researchers specify the task and search spaces at the outset; LLM agents then produce and verify the cycle’s artifacts.

Automated cognitive scientist (AUTOCOG)

  • Researchers provide the task and instructions, experiment and model design spaces, and two seed theories before AUTOCOG begins its autonomous cycle.The experiment design space bounds possible experiments, and the model design space fixes model inputs and outputs.Source: Automated cognitive scientist (AUTOCOG) · p. 3
  • An LLM agent designs an experiment where incumbent theories disagree and proposes a metric; behavioral responses are then collected from humans, foundation models, or cognitive models.The metric quantifies the predicted advantage of one theory over the other.Source: Automated cognitive scientist (AUTOCOG) · p. 3
  • AUTOCOG compares incumbent models’ generative behavior with observations using preregistered metrics, then uses an LLM-agent arbiter to interpret evidence and recommend revisions.Source: Automated cognitive scientist (AUTOCOG) · p. 3
  • A separate LLM agent revises the weaker theory by regenerating its model or its theory and model, then returns it to experimental design.Proposals are programmatically verified and iteratively refined with feedback until valid or the attempt budget is exhausted.Source: Automated cognitive scientist (AUTOCOG) · p. 3
  • The cycle treats discovery as open-ended search, compares generative behavior rather than fitting models, verifies proposals before data collection, and favors accounts that capture accumulated behavior across tasks and populations.The authors describe these principles as supporting a discovery engine that is open-ended, robust, scalable, and unifying.Source: Automated cognitive scientist (AUTOCOG) · p. 3

Results

Across simulated and human decision-making studies, AUTOCOG recovered or surfaced theories that predicted behavior better than its seed theories, including theories that generalized to held-out data. A preregistered follow-up supported the distinctive predictions of its Diminishing Returns WADD theory.

Closed loop recovery of known decision-making strategies

  • 0.013 (SEM = 0.004) MSE ˆp(B) for surfaced theories versus 0.066 (SEM = 0.006) for seeds when recovering canonical strategies without noise.Recovery worsened as action noise increased: aggregate MSE was 0.011 at ε = 0.5 and 0.029 at ε = 0.75.Source: Closed loop recovery of known… · pp. 4–5, Figure 3 · p. 5
  • Five cycles recovered alternating, perseveration, random, and take-the-worst strategies; 20 cycles produced behaviorally aligned theories for single-cue and anti-majority.Manual inspection found exact underlying mechanisms for the first four strategies in at least two of three runs.Source: Closed loop recovery of known… · p. 5

Closed loop discovery of theories in multi-attribute decision-making

  • The human run used five cycles of newly designed experiments and different participants each cycle, totaling 250 participants across ten studies.Theories integrated behavior across experiments, which varied in participants and option-pair designs.Source: Closed loop discovery of theories in… · pp. 5–6
  • 0.093 to 0.0097 MSE ˆp(B) for the best surfaced theory from cycle one to cycle five in the binary-feature human run.A mixed-effects model also found a significant cycle effect on surfaced-theory MSE.Source: Closed loop discovery of theories in… · p. 6, Figure 4 · p. 7
  • The Non-linear Subjective Weighting Model applies a power function to cue validities, with an individual exponent spanning equal-weight, WADD, and Take-The-Best-like behavior.The second surfaced account probabilistically mixes Take-The-Best and a compensatory strategy with transformed validities.Source: Closed loop discovery of theories in… · p. 6
  • Across ten in-run experiments, the non-linear subjective weighting model achieved MSE p-hat(B) of 0.0097 (SEM = 0.0018), significantly outperforming the probabilistic strategy mixture model (M = 0.0482, SEM = 0.0082).On the unseen binary-cue task from Hilbig et al. —24 unique option pairs with four cues of validities 0.9, 0.8, 0.7, and 0.6—its MSE was 0.0377 (SEM = 0.0047), better than TTB (M = 0.0760, SEM = 0.0288), Tallying (M = 0.0784, SEM = 0.0201), and WADD (M = 0.0537, SEM = 0.0111).Source: Closed loop discovery of theories in… · p. 6

Extending AUTOCOG to more complex decision-making tasks

  • 0.018 (SEM = 0.004) for surfaced theories versus 0.138 (SEM = 0.009) for seed theories in MSE ˆp(B) across ten cardinal-rating experiments.Diminishing Returns WADD and Satisficing WADD had similar MSE, but Diminishing Returns WADD was selected as the winner after accounting for near-indifferent predictions.Source: Extending AUTOCOG to more complex… · p. 8, Figure 5 · p. 9

Testing the proposed theory

  • 0.264–0.356 higher match rates for Diminishing Returns WADD than WADD, Tallying, or Take-the-Best on their respective model-discriminating trials.Participants also chose the low-range advantage above chance in steep-versus-flat trials; the level-shift effect was positive but its confidence interval included zero.Source: Figure 5 · p. 9, Testing the proposed theory · p. 10

Discussion

AutoCog’s human-participant theories outperformed their seeds across ten datasets and generalized to a held-out study, while a preregistered study confirmed its novel prediction of diminishing sensitivity. The system searches theory space from behavioral data, but its outcomes remain bounded by researcher-defined design spaces and by the unresolved gap between verbal theories and executable code.

Discussion

  • AutoCog automates theory-building in a loop that searches theories and experiments, gathers behavioral data, diagnoses failures, and synthesizes successor theories.Researchers specify the experiment and model design spaces, seed theories, and behavior source.Source: Discussion · p. 10
  • AutoCog’s discoveries operate at the level of theories—natural-language mechanistic claims paired with executable models—rather than model forms searched over fixed datasets.The discussion identifies this as a distinction from other automated discovery methods [14].Source: Discussion · p. 11
  • AutoCog recovered ground-truth theories across noise levels, including unconventional mechanisms, and converged on random-guessing-equivalent theories for pure noise.Most hand-crafted unconventional strategies were recovered within five cycles; more elusive ones were within reach with additional cycles.Source: Discussion · p. 11
  • In human runs, surfaced theories outperformed both seed theories on all ten datasets, generalized to a held-out study, and predicted diminishing sensitivity confirmed in a preregistered study.The cardinal-feature theory proposed concave valuation of cue magnitudes before reliability weighting; the canonical heuristics predict no such curvature.Source: Discussion · p. 11
  • Each run produces a machine-readable, auditable trace, and researchers specify what counts as a good theory and which spaces the system searches.Running initial searches against a behavioral foundation model could reserve human participants for surviving candidates, conditional on how faithfully that model captures behavior in the target domain.Source: Discussion · p. 11
  • AutoCog searches only researcher-defined experiment and model spaces, does not explicitly score theory virtues, and cannot verify that code faithfully implements its paired verbal theory.The authors also note that novelty emerged rather than being an explicit optimization target.Source: Discussion · p. 11

Methods

The Methods describe AutoCog’s closed-loop process for proposing, testing, evaluating, and revising executable cognitive theories across simulated and human behavior. They also cover evaluation and design controls, data and code availability, and AI-assisted text editing.

Task Domain

  • The task required binary choices between products described by expert ratings, while AutoCog proposed cue count, cue validities, rating scale, and option pairs within fixed task constraints.Ratings carried displayed expert accuracies; the design fixed forced choice, no trial feedback, a maximum of 100 trials, and independent trial randomization.Source: Task Domain · p. 12, Design space · p. 13

AutoCog

  • AutoCog uses two LLM agents to iteratively propose, test, and revise executable theories, with self-verification and logged reasoning supporting autonomy, reliability, and interpretability.Researchers specify essential factors once; simulation-based inference is used for model comparison instead of fitting.Source: AutoCog · p. 12

Pipeline

  • Each cycle proposes theory-targeted experiment–metric pairs, validates discriminability by simulation, collects data, compares predicted and observed outcomes, and revises theories using aggregate loss.Proposals are tested at N = 25 and accepted when a Welch test gives two-sided p < .01; revision candidates are retained for lower mean discrepancy across committed pairs.Source: Pipeline · pp. 12–13

Design space

  • The theory interface fixed prediction and choice-policy functions, while AutoCog generated the decision rule and theory description; candidate theories shared a softmax-and-lapse choice model.All reported runs used Gemini 3.1 Pro Preview with structured JSON outputs; the pipeline also supports other LLM providers.Source: Design space · p. 14, Implementation · p. 14

Ground-truth recovery

  • Recovery tests covered nine canonical and non-canonical strategies, comparing discovered and seed theories with noise-free ground-truth replay using per-stimulus choice proportions, MSE, and correlation.Ground truths included canonical heuristics at ε ∈ {0, 0.5, 0.75} and six non-canonical strategies; scoring replayed theories on 24 option pairs across four binary cues.Source: Baseline theories · p. 14, Ground-truth recovery · pp. 14–15

LLM-as-judge

  • An LLM judge rated whether surfaced theories’ descriptions and code captured ground-truth decision mechanisms, using a 0–1 similarity scale and triplicate averaged scores.The analysis pooled 30 surfaced theories per noise level across TTB, tallying, and WADD ground-truth families.Source: LLM-as-judge · p. 15

Closed-loop theory building with humans

  • Human runs began from task-specific seed theories, iterated the discovery loop, and evaluated surfaced theories using choice-proportion MSE on ten experiments and held-out diagnostic tasks.Binary-task generalization used the fixed stimulus set from [29]; cardinal-task theories were tested against a preregistered study, while cycle trends were assessed with mixed-effects regression.Source: Closed-loop theory building with humans · pp. 15–16

Preregistered follow-up study

  • The preregistered follow-up used two online experiments to test Diminishing Returns WADD against alternative rules and probe whether value advantages matter less at higher values.Experiment 1 recruited 50 participants for model discrimination; Experiment 2 recruited 100 for value-curvature tests, with four features on a 0–5 scale and 96 trials per participant.Source: Preregistered follow-up study · p. 16

Controls

  • LLM-proposed metrics recovered perseveration in 3/3 runs and take-the-worst in 2/3, whereas JSD recovered neither (0/3 each); both metrics recovered TTB in 3/3 runs.The LLM metric uses adaptive, interpretable, per-subject contrasts, allowing between-subject variance diagnostic for perseveration to inform selection. Fixed distances can miss differences their chosen distribution cannot express; lag-1 conditioning captures first-order dependence, while richer sequence distances are computationally costly and constrain experiment design.Source: Controls · p. 17
  • LLM-proposed designs recovered perseveration in 3/3 runs and take-the-worst in 2/3, while random designs recovered neither (0/3 each).Random design paired with JSD recovered TTB in 1/3 runs and neither perseveration nor take-the-worst (0/3 each); LLM-proposed designs recovered TTB in 3/3 runs. The findings indicate better theoretical recovery with LLM-based design, especially with a theory-agnostic metric.Source: Controls · p. 18

Data, software, and code

  • Data, code, and analysis scripts are available online, and generative-AI-assisted text edits were reviewed and edited before use.The reported tools included ChatGPT, Claude, and Writefull.Source: Data, software, and code · p. 18

The AUTOCOG cycle in detail as figures

AUTOCOG’s cycle pits competing executable theories against each other, gathers behavioral data, ranks theories against the accumulated observations, and revises the weaker theory for the next cycle.

Adversarial experimental design

  • Each competing theory proposes an experiment and metric favoring itself; the pair is accepted only after verification shows separation at the planned sample size.This yields one accepted experiment per theory.Source: Figure S1 · p. 22

Behavioral data collection

  • Accepted experiments are run on human participants or simulated behavior, and their option pairs and responses enter a shared observation pool.The same collection pipeline applies across the three behavior sources.Source: Figure S2 · p. 23

Analysis and arbitration

  • The analysis loop forward-simulates every theory against every accumulated observation, characterizes behavior, ranks theories on a shared leaderboard, and gives the cycle to a neutral arbitrator.Source: Figure S3 · p. 23

Theory revision

  • The winning theory remains unchanged while the losing theory is revised or replaced; a candidate is accepted only if it improves fit across the full observation pool.The accepted theory seeds the next cycle.Source: Figure S4 · p. 24

Baseline theories

The baseline set spans frugal, equal-vote, and compensatory decision rules, alongside non-canonical theories designed to challenge recovery across distinct failure modes.

Canonical theories

  • Take The Best uses the highest-validity cue that discriminates, Tallying counts feature wins, and Weighted Additive chooses by validity-weighted feature sums.These rules span one-reason frugality, equal cue votes, and compensatory integration of magnitudes and cue weights.Source: Canonical theories · p. 25

Non-canonical theories

  • Non-canonical baselines test recovery of inverted or extreme frugality, contrarian voting, and stimulus-independent repetition or alternation.Take The Worst follows cues from least to most valid; Single-Cue always uses the least-valid cue; the sequential baselines repeat or switch choices.Source: Non-canonical theories · pp. 25–26

Hyperparameters and budgets

The supplied text identifies a table of fixed AUTOCOG settings used across runs.

Hyperparameters and budgets

Design choices and justifications

AUTOCOG is designed to search openly through agentic generation, with limited researcher input, iterative verification, scalable evaluation, and auditable reasoning.

Design choices and justifications

  • Every generative step is delegated to an LLM agent, avoiding a hard-coded fixed search space of theories.Source: AUTOCOG · p. 26
  • Researchers specify the experimental and model design spaces, seed theories, and analysis metric; the loop then runs without further intervention.Researchers may narrow the search by specifying factors of interest.Source: AUTOCOG · p. 26
  • Experiment and theory proposals are iteratively checked against simulated or observed behavior before acceptance.Source: AUTOCOG · p. 26
  • Forward simulation evaluates theories faster at scale, is harder to game than a single predictive score, and implicitly regularizes against overfitting.The theory must reproduce the observed behavior distribution when simulated, rather than merely fit it afterward.Source: AUTOCOG · p. 26
  • AUTOCOG logs the reasoning behind experiments, metrics, verdicts, and theories, making each run an auditable artifact that another researcher or agent can resume.Source: AUTOCOG · p. 26

Agent prompts

The prompts structure theory discovery around competing, executable accounts: agents design discriminating experiments and metrics, interpret observed results, and use diagnosed failures to guide proposals. Runtime templates supply the task, theories, design, metric, observations, and response format.

Agent prompts

  • Runtime templates fill in the task domain and instructions, executable theory descriptions, experimental design, metric, accumulated observations, and response schema.The generative steps use a fixed system prompt and a templated user prompt.Source: Agent prompts · p. 26

Stage 1 — Experimental design (proposer agent)

  • Each theory advocate proposes an experiment where its own theory should outperform its competitor, targeting quantitative differences in responses to specific stimuli as well as overall performance.The experiment prompt frames the advocate as an adversarial collaborator.Source: Stage 1 — Experimental design… · p. 27

Stage 1 — Metric proposal (proposer agent)

  • Alongside an experiment, an advocate proposes a computable Python metric intended to distinguish the theories on data from that design.The prompt describes the metric as a function receiving a pandas DataFrame and returning a float.Source: Stage 1 — Metric proposal (proposer… · p. 27
  • A proposed metric is admitted only if a Welch two-sample t-test of the theories’ simulated estimates and between-subject variances is significant at the specified human sample size.The prompt cautions that a large mean gap may not suffice when either theory has high between-subject variability.Source: Stage 1 — Metric proposal (proposer… · p. 28
  • Failed metric proposals include their simulated means, between-subject variances, t-statistic, and p-value so the next proposal can diagnose whether its contrast failed through a small gap, high variance, or both.The prompt asks agents to try a qualitatively different metric rather than cosmetically modifying the same idea.Source: Stage 1 — Metric proposal (proposer… · p. 28

Stage 3 — Result interpretation

  • Result interpretation compares the observed metric with both simulated predictions, checks the targeted qualitative pattern, flags confounds, and proposes a next experiment or metric.The prompt asks interpreters to acknowledge ambiguity and avoid overclaiming.Source: Stage 3 — Result interpretation · pp. 28–29

Stage 3 — Arbitration

Arbitration assesses competing theories across experiments, then issues a verdict on whether to revise an existing model or replace a theory. New-theory proposals use arbiter diagnoses, while a programmatic gate retains only candidates with strictly improved aggregate loss.

Stage 3 — Arbitration

  • The arbiter compares theories across all experiments in the domain to identify which better captures the data; if neither explains all experiments, it may propose a new theory rather than a new model.Its structured verdict specifies whether to revise a model or propose a new theory, and which theory is targeted.Source: Stage 3 — Arbitration · pp. 29–30

Stage 4 — Theory revision: new theory

  • A new-theory proposal draws on arbiter diagnoses of mechanistic failures and suggestions for a theory family that may overcome them, with the aim of producing human-like simulated behavior.The prompt directs theory generation toward explanations that work across multiple experiments.Source: Stage 4 — Theory revision: new theory · p. 31
  • The revision loop accepts a candidate only when its aggregate loss strictly improves on the running-best loss; otherwise it rejects the candidate and keeps the existing base.The prompt says the best accepted candidate remains the base for subsequent revisions.Source: Stage 4 — Theory revision: new theory · p. 31
  • Theory parameters use specified string notations, including experiment-defined symbolic identifiers, so model shapes can adapt to experiments instead of being hardcoded.The prompt limits symbolic references to identifiers supplied by the experiment.Source: Stage 4 — Theory revision: new theory · pp. 31–32

Stage 4 — Theory revision: new model

When arbitration selects a new-model revision, the system keeps the theory description fixed and regenerates its executable model components. Candidate revisions pass through a loss-based gate that prevents the running-best model from regressing.

Stage 4 — Theory revision: new model

  • A new-model revision preserves the theory’s prose description verbatim and regenerates only its predict and policy functions and parameter ranges.The revised model is intended to display human-like behavior in experiments in the domain.Source: Stage 4 — Theory revision: new model · p. 33
  • A candidate becomes the running-best model only when its aggregate loss strictly improves; otherwise it is rejected and the base remains unchanged.The loop compares each candidate with the running-best loss.Source: Stage 4 — Theory revision: new model · p. 33
  • Parameter declarations use string notations, including experiment-defined symbolic identifiers, to let parameter shapes adapt to experiments rather than hardcoding them.The prompt permits symbolic identifiers only when they are listed as allowed for the experiment.Source: Stage 4 — Theory revision: new model · pp. 33–34

Stage 4 — Inner critique loop (feedback agent)

The feedback agent judges candidate fit across experiments and recommends continuing or regenerating, while staying within the arbiter’s chosen mechanism family. Separately, an LLM judge scores similarity between surfaced and ground-truth decision rules using a calibrated rubric.

Stage 4 — Inner critique loop (feedback agent)

  • The feedback agent evaluates a candidate against real behavior across experiments and returns either continue or regenerate, with a concrete diagnosis supporting its verdict.The prompt frames the goal as surfacing theories that explain data across multiple experiments.Source: Stage 4 — Inner critique loop… · p. 35
  • Critiques must stay within the arbiter-prescribed mechanism family: the agent may recommend minor adjustments, but not a switch to another family.The agent should also flag when a candidate has drifted from the prescribed family.Source: Stage 4 — Inner critique loop… · p. 35
  • The accept gate keeps only candidates with loss below the running-best loss, and feedback should account for whether earlier advice led to accepted or rejected candidates.A rejected candidate leaves the base unchanged, and the prompt directs the agent to try a different direction rather than repeat rejected advice.Source: Stage 4 — Inner critique loop… · pp. 35–36

LLM-as-judge rubric

  • The LLM judge rates similarity of underlying decision rules on a 0–1 scale, ignoring wording and parameter values; reported pair scores average three ratings per input mode.The two modes use descriptions alone or descriptions together with each theory’s predict() source; fully randomized ground truths are excluded.Source: LLM-as-judge rubric · pp. 36–37

Human experiments

Human experiments used an online platform to recruit adult participants, with consent, per-trial pacing, and preregistered analyses. Table S2 reports preregistered follow-up tests of Diminishing Returns WADD predictions.

Human experiments

  • Experiments were compiled as browser tasks with SweetBean, deployed through AutoRA, and recruited through Prolific, with responses stored in Firebase.Each cycle launched two committed experiments as separate conditions, recruiting 2N participants per cycle.Source: Human experiments · p. 37
  • Studies were conducted under a Princeton University IRB protocol, and adult participants aged 18–55 gave informed consent before the task.Participation was described as minimal risk, with possible mild fatigue or boredom, and participants could withdraw at any time.Source: Human experiments · p. 37
  • Median completion took approximately 6–8 minutes, and participants received US$0.80 per session.Each session had a 20-minute timeout.Source: Human experiments · p. 37
  • Table S2 presents preregistered follow-up tests of predictions generated by Diminishing Returns WADD.Source: Table S2 · p. 38
  • The preregistered tests used one-sided tests in the predicted direction; H1 comparisons were Holm-corrected, and the H3 95% CI included zero despite a significant one-sided test (p = .036).H1 used paired tests; H2 and H3 used one-sample tests. Sample sizes were N = 50 for H1 and N = 100 for H2 and H3.Source: Human experiments · p. 38
  • Participants could not respond until 1,500 ms had elapsed on each trial, a pacing gate rather than a data-exclusion rule.The response keys were locked and the A/B prompt hidden during this interval.Source: Human experiments · p. 38

Surfaced cognitive models from the human-study runs

This section presents executable theories surfaced in human-study runs, including models for binary- and cardinal-valued cues, and describes how the supplied implementation turns feature values into choices while modeling action noise.

Study 1 — Binary-valued cues (seeds: TTB, WADD)

  • The winning Study 1 model uses an individual-specific power transform of cue validities, unifying equal-weight, linear WADD, and Take-The-Best-like strategies.A competing model instead mixes Take-The-Best and flexible compensatory WADD probabilistically, with a power-transformed validity weighting in its WADD component.Source: Study 1 — Binary-valued cues (seeds:… · pp. 38–39

Study 2 — Cardinal-valued cues (seeds: TTB, Tallying)

  • The winning Study 2 model applies concave, shifted power utility to cardinal cue values before validity weighting, discounting extreme advantages relative to moderate gains across cues.A competing satisficing-WADD model binarizes ratings at a threshold, so extreme advantages may be ignored when both options exceed it.Source: Study 2 — Cardinal-valued cues (seeds:… · pp. 40–41

How action noise degrades the recovered mechanism

  • Mechanism similarity to clean Take-The-Best fell from 1.00 at ε = 0 to 0.17 at ε = 1.0, as surfaced theories progressed from exact TTB through noisy TTB variants to near-random guessing.At ε = 0.75, probabilistic stopping retained TTB-like search with judge similarity 0.83; at ε = 1.0, the model abandoned cue-based search.Source: How action noise degrades the… · pp. 41–44
  • The minimal-effort choice rule scores each option by summing its feature values, then applies a softmax to obtain choice probabilities.The softmax subtracts the maximum score before exponentiation for numerical stability.Source: How action noise degrades the… · p. 44
  • When the lapse rate approaches 1, the model’s action noise is dominated by uniform responding.The implementation mixes the core choice probabilities with a uniform distribution across options.Source: How action noise degrades the… · p. 44

Related papers

Loading 2606.26448v1…