PaperlayerView exact revision on arXiv

Source-linked AI summary

Programs-of-Layers in LLMs through the Lens of Cortical Areas

Justus Westerhoff, Stephan Olbrich, Hatem Oraby, Matthew Evan Larkum, Felix Alexander Gers

arXiv:2609.31360v1cs.AI

TL;DR

Transformer inference ordinarily runs every layer in a fixed pass, raising the question of whether input-dependent computation can recover latent capability without retraining. This paper reconstructs and evaluates PoLar’s search and learned router, finding that dynamic layer programs can outperform the standard pass even though the router’s top prediction collapses to it. The analysis also examines program coverage and brittleness, while noting that router evaluation on out-of-distribution data remains unavailable.

  • Problem

    A standard transformer forward pass performs fixed computation regardless of input difficulty, leaving open whether inference can adapt that computation to recover latent capability without retraining.

    Source: §1 · p. 1

  • Method

    The authors reconstruct PoLar’s MCTS search over layer edits, evaluate it across models, and audit the programs and learned router that predicts them.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    Dynamic skipping or repeating outperformed the fixed pass, but the router’s top prediction collapsed to the default pass across tested models and settings, while MCTS found better programs for nearly every solvable question.

    Source: §1 · p. 3, §6 · p. 13

  • Takeaways & Limitations

    A small menu of generic programs covers most questions found by a larger search, but programs that correct errors are brittle to undoing even one edit.

    Source: §1 · p. 3

    The trained router could not be evaluated on out-of-distribution data because its top prediction always collapsed to the default forward pass.

    Source: §6.1 · p. 13

Abstract

from arXiv · show

Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human brain does not work this way: using the thalamus as a central hub, it routes information flexibly to all regions of the cortex according to demand. Li et al. (2026) recently showed, with a system they call program-of-layers (PoLar), that transformers can be given an analogous flexibility if their layers are treated as a library of functions rather than a fixed sequence. Performance improves over the standard forward pass when each input is dynamically routed through an adaptive sequence of skipped or repeated contiguous layer blocks. We reconstructed PoLar's diagnostic MCTS in more detail than the original paper and applied it across 5 models. We reproduced several of PoLar's findings: skipping outperformed the standard pass, repeating outperformed skipping, and combining both outperformed either alone. Shorter programs sufficed for easier questions, while harder questions required more layer repeats. However, we failed to replicate the main claim regarding their learned router for single-shot inference: its top-ranked prediction consistently collapsed back to the standard pass, even though its top-k predicted programs, taken together, did show a real accuracy gain. Beyond reproduction, we find that a small number of generic programs are enough to solve most of the questions. We also provide a much deeper analysis of these programs' structure and robustness: for example, we found that programs that correct errors are highly brittle: undoing even a single edit inside a program typically breaks the correction. Connecting this to the brain's routing mechanisms, PoLar mirrors principles of thalamo-cortical coordination between cortical-area-like transformer layers. We publicly release the code at https://datexis.github.io/RE-PoLar/

1 Introduction

The paper asks whether fixed-pass transformer inference can adapt its computation to recover latent capability, drawing an analogy to the brain’s context-sensitive routing. It reproduces and audits PoLar, finding that diagnostic search yields useful programs while the learned router’s top prediction collapses to the standard pass.

1 Introduction

  • PoLar’s coordinating role is presented as functionally analogous to thalamic routing between cortical areas.The analogy motivates treating transformer layers as modules whose execution can be routed at inference time.Source: Figure 1 · p. 2, §1 · pp. 2–3
  • The motivating question is whether a fixed transformer forward pass can be adapted at inference time to recover more of a model’s latent capability without retraining.The paper contrasts fixed computation per pass with the brain’s adaptive allocation of neural computation across tasks.Source: §1 · pp. 1–2
  • The authors frame PoLar as a function library that replaces a fixed layer pipeline with input-specific schedules of skipped, kept, or repeated layer segments.They reconstruct the diagnostic MCTS and router study across five models and release code and search implementation.Source: §1 · pp. 2–3
  • Across five models, the router’s top-1 prediction remained the standard forward pass in every tested configuration, so the authors could not reproduce its headline result.They reconstructed the diagnostic search and report reproducing most of PoLar’s other findings.Source: §1 · p. 3
  • A small menu of programs covers most of the questions found by a much larger search, but undoing one edit in a program that fixes an error usually breaks the fix.The candidate pool contains executable alternatives, but the router does not rank the right candidate first.Source: §1 · p. 3

2 Motivation

The paper motivates PoLar through parallels between flexible brain routing and transformer layer reuse. Preliminary evidence suggests that layer similarity boundaries correspond to which contiguous blocks tolerate skipping or repeating.

2.1 Biological relevance

  • The authors draw a functional analogy between transformer layers and cortical columns, with the thalamus coordinating which cortical-like modules engage and how information flows.They note that cortical computation runs in parallel across columns, with only a relevant subset active at a time.Source: §2.1 · pp. 3–4

2.2 Preliminary Evidence for Layer Redundancy

  • On Qwen3-8B’s six-domain MMLU-Pro subset, repeating layers 18–22 improved accuracy; layer 6 marked a boundary where mixed blocks collapsed accuracy, while skipping past layer 25 was roughly neutral.The representational and behavioral patterns aligned: middle layers 18–22 benefited from repetition, and later layers were mutually redundant.Source: Figure 2 · p. 4, §2.2 · p. 5
  • The preliminary result motivates asking whether systematic search or a lightweight predictor can find the best per-input program over layer functions.The layer-block interventions are described as a single-model, single-benchmark, single-operator instance of that question.Source: §2.2 · pp. 4–5

3 Methodology

The method represents each input’s computation as a program over pretrained transformer layers, then uses MCTS to search structured skip/repeat edits and trains a router to predict programs. The authors also detail the search reconstruction and training-target choices.

3.1 Program-of-Layers Formalism

  • A PoLar is an arbitrary finite sequence of layer indices whose composition replaces the standard pass; a program is rescuing when it fixes an input that Base gets wrong.The search space is restricted to contiguous segments, each assigned skip, keep, or repeat, with segments capped at four layers.Source: §3.1 · p. 5

3.2 Search Procedure through MCTS

  • MCTS nodes are canonical sets of non-overlapping skip/repeat edits on the identity program, with gaps kept by default; the search expands under a widening cap or descends by UCB once capped.Each candidate is directly executable, and the widening cap encourages exploration beyond single-edit root children.Source: §3.2 · pp. 5–6
  • The reconstructed MCTS scores each landed-on program with one model call, backs up binary correctness reward along the visited path, and increments a global simulation counter.The authors specify choices left underspecified by the source papers, which did not release search code.Source: §3.2 · pp. 5–6

3.3 Router: Architecture and Training-Target Construction

  • The router predicts segment boundaries and operations from frozen input embeddings and per-layer queries, then uses a small beam search to rank candidate programs.Training uses valid MCTS programs, caps examples at 50 per question, drops repeats not matching two executions, and reweights identity programs and questions with many examples.Source: §3.3 · pp. 6–7

4 Experiments and Results

Across six frozen models, the experiments reproduce several MCTS findings that skipping and repeating layers can improve accuracy, while the learned router’s top prediction collapses to the standard pass. Additional analyses find broad menu coverage, fragile error corrections, and answer-position overfitting on MMLU-Pro.

4.2 MCTS Diagnostic Findings vs. PoLar

  • Skip-and-repeat programs outperform Base at every tested model and DART-Math difficulty, with repeat alone stronger than skip alone.The reported ordering is Base < Skip-only < Repeat-only < Skip&Repeat.Source: §4.2 · p. 8
  • Most valid programs are shorter than the standard forward pass on four of five models, closely matching that finding on the fifth.The result is shown for Qwen3-8B in Figure 4, with per-model breakdowns in appendix figures 13 and 14.Source: §4.2 · p. 9, Figure 4 · p. 9
  • Harder DART-Math questions more often need recurrence, while the share needing skipping rises on four of five models.For Qwen3-32B, easier questions concentrate skips in fewer layer positions, whereas harder questions spread repeats across more of the stack.Source: §4.2 · p. 9, Figure 5 · p. 9, Figure 6 · p. 9
  • Across the five models, 73–86% of valid-program segments use at most one recurrence, but four models fall below both of Li et al.’s reported segment-length thresholds.The thresholds reported by Li et al. are 54.5% of segments touching one layer and over two-thirds touching at most two.Source: §4.2 · p. 10, Figure 7 · p. 10

4.3 Beyond PoLar: Menu Generalization, Error Analysis, and Program Robustness

  • A small fixed menu of non-identity programs covers most questions, and its top 25 programs edit diverse layer bands rather than sharing one recipe.Each top-25 program edits one contiguous block; edits cluster away from early and later bands around layers 6 and 25.Source: §4.3 · p. 10, Figure 8 · p. 10
  • Related wrong answers are rescued by valid programs roughly 1.5× more often than unrelated wrong answers in every model.The comparison uses an LLM-assisted classifier to assess how close each wrong answer is to the ground truth.Source: §4.3 · pp. 10–11
  • Reverting one edit breaks a genuine rescuing program’s reward in 52.5%–72.3% of cases, with fragility varying more by model than difficulty.Programs average 2.01 edited segments; the figure pools single-segment reversions.Source: §4.3 · p. 11, Figure 9 · p. 11

4.4 Router Reproduction: Identity-Collapse

  • The router’s top-1 prediction is always identity, while its top-5 can retain an accuracy gain through beam-search diversity; MMLU-Pro rescuing programs also overfit answer position.Identity examples account for only 2.4–7.0% of training examples, and reshuffled MMLU-Pro rescuing accuracy falls below identity’s own.Source: §4.4 · p. 11, Table 3 · p. 11, §4.5 · pp. 11–12

5 Related Work

Related work spans methods that save compute by dropping or routing layers and methods that reuse computation through recurrence. PoLar differs by routing whole programs through frozen pretrained layers, and the paper frames this global coordination through a thalamo-cortical analogy.

5 Related Work

  • Layer-dropping and early-exit or interior-layer routing methods mainly aim to reduce inference compute, while recursive transformer approaches reuse layers to add computation.The cited approaches differ in whether layers are removed, selected per input, or reused through loops.Source: §5 · p. 12
  • PoLar routes a whole program through a frozen pretrained model without architectural change, unlike methods that modify or train the model’s routing structure.DR.LLM learns local per-layer policies and cannot express PoLar’s multi-layer block repetitions.Source: §5 · p. 12
  • The paper interprets PoLar’s global choice of which layers engage, in what order, and when to loop as a thalamus-like coordinating role.The contrast is with mixture-of-experts routing, which decides locally per layer and token.Source: §5 · p. 12

6 Conclusion

The study reproduces the benefit of dynamically skipping or repeating transformer layers, but not the router’s top prediction, which returns the default pass. It identifies router evaluation on OOD data and closer reproduction of prior MCTS numbers as unresolved limitations.

6 Conclusion

  • Dynamically skipping or repeating layer operations outperforms the standard fixed-depth pass, but the router’s top prediction collapses to that default despite MCTS finding better programs.The authors attribute the collapse to a fractured program-space error surface, making single-shot deterministic prediction a poor strategy.Source: §6 · p. 13

6.1 Limitations and Future Work

  • The trained router could not be evaluated on OOD data because its top prediction always collapsed to the default pass.The authors suggest a router that weighs several candidate programs at once as a possible direction for addressing both issues.Source: §6.1 · p. 13

A.1 Dataset construction: our DART-Math split

The authors reconstruct DART-Math’s five difficulty levels because the discrete split is not released, aiming to make difficulty-wise results reproducible. Their split matches reported level counts, but may not match Li et al.’s exact split.

A.1 Dataset construction: our DART-Math split

  • The authors release a script to deterministically rebuild the split rather than redistribute it, because part of its source pool traces to disputed competition-math content.The GSM8K portion has no such issue.Source: §A.1 · p. 19
  • The reconstructed difficulty split follows ascending fail rate, divides the pool into five near-equal bins, and assigns them DM-1 through DM-5.Fail rate is defined as one minus the pass rate measured by sampling DeepSeekMath-7B-RL.Source: §A.1 · p. 19
  • Each difficulty bin uses a seeded shuffle and contributes 1,250 train, 250 validation, and 500 test queries, totaling 10,000.The counts match Li et al.’s reported 5 × 2,000 queries, but the authors cannot guarantee an identical split.Source: §A.1 · p. 19

A.2 Prompt sensitivity in evaluation

Prompt wording can trigger two distinct answer-format failures, while prompt interventions reduce them enough to support the paper’s program-level analyses. The authors also find that applying native chat templates lowers accuracy in their tested ablation.

A.2 Prompt sensitivity in evaluation

  • Under this exact wording, 43.8% of Qwen3-8B Base generations were prompt-template echoes: the model emitted \boxed{ANSWER} verbatim instead of a real answer.The extraction code takes the last \boxed{...} span, so a later echo determines the grade when one occurs.Source: §A.2 · p. 19
  • Unboxed outputs are distinct from placeholder echoes and dominate polar_default on three of six models.The evaluation reports the two failure modes in separate columns because accuracy alone conflates them.Source: §A.2 · p. 19
  • The authors swept six prompt wordings across six models, including DR.LLM-based variants, response prefill, and a one-shot demonstration.The final sweep followed a smaller pilot that identified prefill and a trivial demonstration as effective interventions.Source: §A.2 · pp. 19–20
  • On Qwen3-8B, polar_oneshot reduced the unboxed rate from 91.2% to effectively 0% without exceeding the 50-token budget.Both interventions significantly improved over polar_default on four of six models; Qwen3-32B remained at 38.6% unboxed under polar_oneshot.Source: §A.2 · p. 20
  • The authors use polar_oneshot for program-level analysis, while noting that Qwen3-32B’s 38.6% unboxed rate may contaminate rescuing-program counts.The prompt improves accuracy over polar_default on every model except a tie on LLaMA-3.2-3B.Source: §A.2 · p. 20

A.3 Decision against chat templates

  • With prompt content fixed at polar_oneshot, applying the native chat template lowered accuracy on all six models in the 200-question ablation.The authors therefore use no chat templates throughout, deliberately deviating from Li et al.’s released mechanism for some models.Source: §A.3 · p. 20

A.4 Full baseline reproduction breakdown

The section reports baseline accuracy across DART-Math, MMLU-Pro, and two out-of-distribution transfer targets, while documenting model-dependent reproduction gaps. OOD gaps broadly track the DART-Math gaps by model, without a proposed mechanism.

A.4 Full baseline reproduction breakdown

  • Table 7 extends Table 2 with p@2–p@4, LLaMA-3.2-3B, two OOD transfer targets, and MMLU-Pro, all under polar_oneshot with no chat template.Source: §A.4 · p. 20
  • Table 8 reports Base accuracy on all 14 MMLU-Pro categories and on ASDiv and MAWPS under the table 7 setup.MMLU-Pro uses forced-choice logit scoring; ASDiv and MAWPS use generation-based scoring.Source: §A.4 · p. 21
  • The reported OOD comparisons include matching Li et al. differences for ASDiv and MAWPS, but no aggregate MMLU-Pro difference.Li et al. report only subject-level MMLU-Pro results, covering 13 of the 14 categories evaluated here.Source: §A.4 · p. 21
  • Every ASDiv/MAWPS accuracy difference was positive except for LLaMA-3.2-3B on both benchmarks and Qwen1.5-MoE-A2.7B on ASDiv.The same two models mostly ran below Li et al.’s results on DART-Math, and the direction of the DART-Math gap broadly tracked the OOD gap by model.Source: §A.4 · pp. 21–22
  • Table 6 describes a 200-question-per-model chat-template ablation with polar_oneshot content held fixed.The accompanying results passage states that applying the template never improves accuracy and has a negative effect on all six models.Source: Table 6 · p. 22

A.5 Dropping LLaMA-3.2-3B

The appendix explains why LLaMA-3.2-3B is excluded from the search and router track, details key MCTS reconstruction choices, and describes batched decoding used to accelerate reward evaluation. It also reports a production-scale speedup for the KV-cached implementation.

A.5 Dropping LLaMA-3.2-3B

  • LLaMA-3.2-3B is excluded from MCTS and router training/evaluation because its Base reproduction remains poor after two evaluation fixes.Its accuracy does not vary monotonically with nominal DART-Math difficulty, and it undershoots published numbers almost everywhere.Source: §A.5 · pp. 22–23

A.6 MCTS design choices: what we changed from a literal reading of the papers, and why

  • With 276 single-edit actions but 200 simulations on Qwen3-8B, progressive widening prevents the root from spending the entire budget on single-edit siblings.The widening cap permits discovery of multi-edit programs, including combined skip-and-repeat programs.Source: §A.6 · pp. 24–25
  • The search allows edits in any order to remove placement bias, but disallows overlapping edits to keep its programs representable by the downstream router.The earlier cursor-constrained search overrepresented late-stack placements under constant reward.Source: §A.6 · p. 25
  • The MCTS uses a shared global simulation counter in its UCB score to match Li et al.’s stated formula, despite the textbook-local alternative.The authors do not claim the global-counter choice has the local UCT variant’s per-node regret-bound guarantee.Source: §A.6 · p. 26
  • The reconstructed MCTS uses hierarchical descent because flat whole-tree selection produced deep trees with median edit-count 10 and maximum 26.Those depths conflict with the finding that valid programs are predominantly short; hierarchical descent is the default.Source: §A.6 · p. 26
  • DR.LLM’s search statistics are not directly comparable because it exits early and uses continuous partial credit rather than binary correctness.Early exit structurally undercounts equally good alternative programs, while the reward difference changes what counts as valid.Source: §A.6 · p. 26

A.7 Accelerating MCTS reward computation: masked-batch decoding and its Key-Value (KV)-cached extension

  • Masked-batch decoding groups rows by their next required layer, sharing layer forward calls across distinct programs to avoid launch-bound single-row generation.A largest-group-first schedule trades some waiting by early-ready rows for fewer, larger forward calls.Source: §A.7 · p. 27
  • On Qwen3-8B’s 2,000-question DART-Math difficulty-1 split, KV-cached decoding was 1.10× faster (8h14m vs. 9h04m).Solved counts differed by one question (1,776 vs. 1,777), and shorter-than-identity rates differed by 0.0012.Source: §A.7 · p. 27

A.8 Router training recipe, architecture, and the identity-collapse investigation

The authors reconstruct the router’s training and evaluation pipeline, then investigate why its top-ranked programs remain ineffective. Their audit finds that the router can produce non-identity outputs, but does not reliably rank correct alternatives first.

A.8 Router training recipe, architecture, and the identity-collapse investigation

  • 53.3% of discovered repeat-containing programs include a segment repeated other than twice, conflicting with Li et al.’s qualitative finding; the discrepancy remains unresolved.The authors flag differing search dynamics or aggregation as possible explanations.Source: §A.8 · p. 29
  • Of 1,105 repeat-containing training pairs reverified after cropping, 758 (68.6%) remained correct and 347 (31.4%) were discarded.The Crop treatment retains roughly twice as many programs as Drop, but does not check whether the shortened program remains correct.Source: §A.8 · p. 29
  • At λ=1.0, 3.6% of test questions receive non-identity top-1 predictions, but real pass@1 is 49.6% versus Base’s 50.8% (−1.2pp).At λ=2.0, non-identity predictions reach 40.0% while accuracy falls by 16.4pp; router pass@1 never exceeds Base across tested λ values.Source: §A.8 · p. 31
  • Even when the router produces diverse candidates, rescue@1 stays at zero while rescue@5 is substantial, including on training data.The authors suggest positive-only supervision may explain why the router does not rank valid rescuing programs above incorrect candidates.Source: §A.8 · pp. 31–32
  • Drop has the lowest validation loss within both CE and focal families across every tested model-and-difficulty combination, and Drop-CE is selected for the remaining paper.Across 40 configurations per loss family, real pass@5 differs by only 0.06pp: 41.96% for CE versus 41.90% for focal.Source: §A.8 · p. 32, Table 11 · p. 32
  • The router’s top-5 candidate sets contain at most 22 distinct programs per model-and-difficulty cell, with the same small menu across training, validation, and test splits.The table reports counts out of up to 500 possible programs per split.Source: §A.8 · pp. 32–33, Table 12 · p. 33

B.1 Compute infrastructure and hardware

  • Reported search and evaluation used roughly 2,000 GPU-hours, excluding development and debugging; error-tagging and solver-check use added roughly 60M tokens on separate infrastructure.Search and evaluation runs were pinned to one GPU architecture because bf16 non-associativity can cause greedy decoding to diverge across architectures.Source: §B.1 · p. 33

B.2 MiniMax-M2.7-AWQ as LLM-assisted error tagger

The paper describes an LLM-assisted error-tagging procedure and reports how error type relates to whether search finds a fixing program. It also reproduces skip-and-repeat gains and adds further program analyses.

B.2 MiniMax-M2.7-AWQ as LLM-assisted error tagger

  • The error taxonomy is derived mechanically from boxed-answer extraction; the tagger labels closeness but does not determine correctness.Across the full corpus, 88.2% of errors are wrong final answers; unboxed outputs account for 59.3% of Qwen3-32B’s errors, versus at most 1.1% for each other model.Source: §B.2 · pp. 33–34, Table 14 · p. 35
  • Programs rescued numerically close wrong answers more often than structurally different ones, with rescue rates of 57.9–83.1% versus 36.9–62.7%.The closeness label is assigned to wrong final answers by MiniMax-M2.7-AWQ, shown only the model answer and ground truth; the strict grader determines correctness.Source: §B.2 · pp. 33–34, Table 15 · p. 35

C.1 Li et al.’s figure reproductions

  • Skip&Repeat gains over Base range from +31.7 to +68.7 accuracy points across all five models and every DART-Math difficulty.The reproduced comparison defines Skip&Repeat as identity or any valid program, and Base as identity already correct.Source: §C.1 · p. 35, Figure 10 · p. 35, Table 16 · p. 36

C.2 Results with no PoLar equivalent

Additional analyses detail how discovered programs vary by difficulty, operation, and layer position, while an MMLU-Pro analysis finds that answer-key letter frequencies are close to uniform. The latter provides a baseline for interpreting position sensitivity in programs.

C.2 Results with no PoLar equivalent

  • Menu coverage continues climbing at K=100 on the hardest difficulty but is nearly flat by K=30 on the easiest, across the per-model panels.The figure plots solved-question coverage against menu size K on a log scale.Source: Figure 12 · p. 37
  • The appendix breaks down, by model and difficulty, whether questions are solved by identity, skip-only, repeat-only, or combined programs; these checks are independent and need not sum to 100%.It also shows the baseline, rescued, and unsolved shares for each difficulty.Source: §C.2 · p. 37, Figure 16 · p. 38, Figure 15 · p. 38
  • Per-layer figures extend the skip and repeat analyses to additional models, showing skip and repeat shares by layer index and difficulty.The skip figure notes that earliest layers are almost never skipped; Qwen3-32B is omitted because it has a dedicated main-body figure.Source: §C.2 · p. 37, Figure 17 · p. 38, Figure 18 · p. 39

D Program-of-Layers on MMLU-Pro

  • MMLU-Pro answer keys show no discrete-letter bias across checked option-count groups, and the dominant 10-option group is close to uniform at 9.4–11.3% per letter versus a 10.0% random baseline.The dominant group contains 4,220 of 5,083 training rows (83%).Source: D Program-of-Layers on MMLU-Pro · p. 39

D.1 Option-order robustness: MCTS-discovered programs track position, not content

The authors test whether MCTS-discovered programs depend on answer content or option position by reshuffling choices while leaving question content unchanged. Rescuing programs lose most of their accuracy after reshuffling, unlike the identity control.

D.1 Option-order robustness: MCTS-discovered programs track position, not content

  • The test re-scores MCTS-discovered programs after randomly reordering options without changing question content, then compares accuracy with the original order.Qwen3-8B’s full 12,032-query search is also checked to rule out an artifact of train/validation/test subsetting.Source: §D.1 · p. 39
  • On 6,866 identity-wrong queries rescued in the original arrangement, MCTS programs’ accuracy falls from 94.1% to 17.7% after reshuffling, a 76.4-point drop.Identity accuracy remains nearly unchanged within noise, 42.2% → 41.9% across 12,032 queries, despite substantial per-query answer changes.Source: §D.1 · p. 39
  • Reshuffle fragility also appears among programs for identity-correct queries, whose accuracy drops from 96.8% to 36.6%; rescuing programs fall further, from 94.1% to 17.7%.The authors conclude that measured non-identity programs track option arrangement, with programs built to correct identity failures most dependent on the original arrangement.Source: §D.1 · p. 40

D.2 Router reproduction on MMLU-Pro: a random-valid-program control

On MMLU-Pro, the router’s top-1 prediction collapses to identity, while its pass@5 margin over random programs remains structurally fixed. The related discussion distinguishes path-level coordination from other brain-inspired approaches, documents reconstruction limits, and discloses LLM-assisted development.

D.2 Router reproduction on MMLU-Pro: a random-valid-program control

  • 43.9% identity accuracy accompanies top-1 identity collapse across all 2,800 held-out MMLU-Pro queries; Drop-CE reaches 62.6% pass@5 versus 45.3% ± 8.8% for random programs.The same top-five candidates recur across five independently seeded retrainings, so the margin is not per-question adaptation.Source: §D.2 · p. 40, Table 18 · p. 41

E Further Brain-Inspired Machine Learning Approaches

  • PoLar coordinates the whole layer execution path, unlike locally scoped MoE routing; the discussion also notes brain-inspired redesigns and an unrun MOT test of layer-to-cortex alignment.MOT offers a concrete way to test whether revisited model layers align with cortical regions, an experiment the authors only gesture toward.Source: E Further Brain-Inspired Machine… · p. 41

F Communication with the Original Authors

  • The released repository covers router training and evaluation with precomputed MCTS programs, so the authors independently reconstructed the search, difficulty split, and baseline-table code.Their attempts to contact Li et al. about unpublished MCTS components were unsuccessful.Source: F Communication with the Original… · p. 42

G Use of Large Language Models

  • The authors used Claude Code to develop experiment code and draft or edit prose, then reviewed and edited generated material and verified reported claims and numbers.The authors state that all reported numbers, claims, and conclusions are their own.Source: G Use of Large Language Models · p. 42

Related papers

Loading 2609.31360v1…