MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability

1 Introduction

• Most existing benchmarks test answer extraction or formula repair in otherwise finished workbooks, missing modeling decisions, formatting, and layout required to build from scratch.The related-work discussion notes that SpreadsheetBench 2 also derives tasks from complete workbooks with parts removed and largely grades computed values.Source: §1 · p. 2, Related Work · p. 4
• The paper identifies two challenges: assembling broad, publicly releasable financial tasks and evaluating readability and auditability beyond accuracy at scalable cost.Expert pairwise judgments can be costly to scale and difficult to reproduce.Source: §1 · pp. 2–3
• MBABench-2.0 releases 101 construction tasks across key financial fields and evaluates spreadsheet quality with 129 criteria in 12 categories.The paper also develops and validates an agentic judge for scalable evaluation.Source: §1 · pp. 3–4
• The top agent passes only 16% of tasks, and agents fall short in numerical accuracy and other spreadsheet-quality criteria.Errors include misreading instructions and producing models that are difficult to modify or audit.Source: §1 · pp. 3–4

Related Work

• Prior work spans spreadsheet manipulation and construction benchmarks, while this work combines a public construction benchmark with a taxonomy and validated agentic judge for quality evaluation.The paper positions its approach alongside spreadsheet benchmarks using automatic graders and LLM-based judges.Source: Related Work · p. 4

2.1 Difficult and Long-Horizon Financial Spreadsheet Tasks

• 50 of 101 tasks are Medium-Hard or Hard, and estimated build times reach 33 hours for a modeler with 3+ years of experience.The task set spans 16 task types and seven financial fields, and the full tasks and golden solutions are released.Source: §2.1 · p. 5
• Tasks were created by finance experts from scratch, refined to remove identified ambiguities, and released with input workbooks and golden solutions.Each attempt must answer all task questions correctly to satisfy the Final calculation accuracy criterion; tasks underwent multi-stage human and LLM review.Source: §2.1 · p. 5
• Figure 2 plots the 101 tasks by modeling-skill difficulty and estimated build time, with about half rated Medium-Hard or Hard.The benchmark spans levels 2–5 of a six-level difficulty scale.Source: Figure 2 · p. 5, §2.1 · p. 5
• Figure 3 compares task-type coverage: MBABench-2.0 covers more types than BankerToolBench and GDPval, and releases 101 tasks, including 50 Medium-Hard+ tasks.MBABench-1.0 is comparable in coverage but released 38 tasks, including 6 Medium-Hard+ public tasks.Source: §2.1 · p. 5, Figure 3 · p. 6

2.2 An Illustrative Example - DayMonthYear

• DayMonthYear requires an integrated three-statement model over 3,653 days, monthly DCF valuation, and answers to 130 questions, with an estimated build time of about nine hours.Its requirements include daily-to-monthly and yearly consistency, weekday cost rules, date-specific events, valuation steps, and a circular WACC-equity relationship.Source: §2.2 · pp. 6–7
• The challenge is satisfying interdependent requirements across thousands of days from inputs alone, rather than repairing a few formulas in a workbook whose structure is already provided.An early error can propagate across later days, while WACC and equity value depend on one another.Source: §2.2 · pp. 6–8, Figure 4 · p. 7

• MBABench-2.0’s rubric applies universal criteria to all tasks and supplements them with task-dependent conditional criteria, for 129 criteria total: 93 universal and 36 conditional.Source: §3 · p. 8
• House standards specify conventions left open by public modeling codes, while the rubric itself remains undisclosed to agents.The supplied guidelines make conformance checkable against a known specification and comparable across agents.Source: §3 · p. 8
• Each task’s applicable criteria share category weights, supporting a consistent scale and criterion-by-criterion judge testing.New tasks require relevance decisions for conditional criteria, and new model types require only their specific criteria.Source: §3 · pp. 8–9
• The Single date driver criterion requires one labeled anchor date to drive time periods and dependent calculations in time-based models.Figure 5 illustrates a named ValDate driving DCF period ends, stub periods, mid-year discount periods, and roll-forwards.Source: §3 · p. 9, Figure 5 · p. 9
• Final calculation accuracy and 19 other criteria are checked deterministically; an agentic judge grades the remaining 109 context-dependent criteria.The judge must accommodate acceptable spreadsheet layouts, such as units placed one to three cells from their numbers.Source: §3 · p. 9
• A pass requires correct final calculations and no more than k_i violations at each importance level; tolerated violations account for judge false positives.The k_i values are chosen to weight practical criterion importance and give a perfect workbook a high probability of passing. Because the rule is lenient and the judge has a higher false negative rate, pass rate is an upper bound on acceptable workbooks.Source: §3 · pp. 9–10

4 Judge Validation

• Validation stress-tests the judge on controlled examples for individual criteria and on evaluated agents’ actual attempts.The real-attempt tests examine performance on the attempt distribution seen at deployment.Source: §4 · p. 11

4.1 Per-Criterion Controlled Validation

• Across 654 controlled verdicts, the judge reaches 98.2% accuracy, 0.31% FPR, 3.4% FNR, and 98.1% F1.Tests covered all 109 judge-graded criteria using 218 clean or manually altered examples, each graded three times; 207 were correct in all runs and 217 in at least two.Source: §4.1 · p. 11

4.2 Validation on Agent Attempts

• On 1,152 rubric determinations across 12 expert-annotated attempts, the judge achieves 96.4% accuracy, 1.82% FPR, 11.1% FNR, and 90.4% F1.The 12 attempts came from four agents spanning a range of capability.Source: §4.2 · p. 12
• The same verdict appeared in all three runs for 90.0% of attempt-criterion pairs, near the 89.7% expected under independent errors; pass verdicts disagreed in 1.5% of pairs.Across 90 attempts, the standard deviation in score across repeated gradings was 3.0 out of 100.Source: §4.2 · p. 12
• With Sol- and Fable-based judges, all three tested agents keep their relative score and pass-rate rankings, and scores correlate at 0.901 across attempts.The comparison regraded 303 attempts.Source: §4.2 · p. 12
• When the judges disagree, Fable flags a violation in 60.0% of assessments and scores attempts 1.91 ± 0.39 points lower on average.The authors report this as consistent with earlier qualitative observations that stronger judge models flagged more violations.Source: §4.2 · p. 12
• The score gap between In-house Fable 5.1 and Codex GPT-6 Astra is statistically insignificant between the Sol and Fable judges: 0.30 ± 0.65 across 303 attempts.The reported gaps are 11.28 points under Sol and 11.58 under Fable.Source: §4.2 · p. 12

5.1 Quantitative Analysis

• Most attempts miss the pass bar, and pass rates change drastically across harnesses even for the same underlying model.The authors conclude that the harness or product built around the model has a major effect on spreadsheet quality.Source: §5.1 · p. 13
• 59% accuracy on Medium-and-harder tasks and 45% on Hard tasks mark the top agent’s numerical performance; accuracy is more stable across commercial harnesses than pass rate.Figure 8 also shows accuracy decreasing as tasks get harder.Source: §5.1 · p. 13, Figure 8 · p. 14
• Solve time generally rises with task difficulty, and agents whose time rises least also perform worst, suggesting proficient agents may recognize difficulty and spend more effort accordingly.Agents use far less wall-clock time than estimated human build times, but the authors caution that this must be weighed against incorrect answers.Source: §5.1 · p. 13
• Scores averaged 90.1 at Max, 88.3 at High, and 86.9 at Low reasoning effort for Claude Code (Fable 5.1).The authors report a positive correlation between reasoning level and score.Source: §5.1 · p. 13

5.2.1 Final Calculation Accuracy

• Calculation failures include instruction misreading (44%), poor professional judgment (28%), delivery or technical failures (15%), and implementation errors (11%).Examples include double-counting a monthly salary on a bi-weekly schedule, leaving answer cells blank, and dropping a loan’s final coupon.Source: §5.2.1 · pp. 13–14
• 85% of frontier agents’ erroneous attempts involve misreading or poor judgment, versus 52% for non-frontier agents; building or delivery errors account for 13% versus 47%.Frontier agents make fewer errors building or delivering models, but not fewer misreadings or judgment errors.Source: §5.2.1 · p. 15

5.2.2 Other Quality Criteria

• Models can be hard to reuse when timelines are fixed, and harder-task formulas can become difficult to audit; Excel add-ins often omit reviewer aids such as named inputs and assumption sources.On DayMonthYear, 11 of 14 agents outside the in-house harness violated Time-period flexibility.Source: §5.2.2 · p. 15
• Sense checks may only confirm internal consistency, while even the top-ranked agent uses same-workbook comparisons, unnamed key inputs, and thousands of inactive formula rows or columns.A master error flag can display “OK” while ignoring error values; presentation conventions such as freeze panes and color coding are also inconsistent.Source: §5.2.1 · pp. 14–15, §5.2.2 · pp. 15–16

6 Conclusion

• MBABench-2.0 is a fully public benchmark of 101 diverse, difficult financial spreadsheet construction tasks developed over approximately 2,095 person-hours.The paper also introduces 129 criteria across 12 categories and a corresponding agentic judge validated against human judgment.Source: §6 · p. 16
• Across 20+ agents, the best-performing agent passes only a small fraction of tasks, with shortcomings in both numerical accuracy and high-level spreadsheet quality.The paper characterizes these failures across accuracy and the quality criteria in its taxonomy.Source: §6 · p. 16
• The authors conclude that modern agents still fall short of delivering high-quality spreadsheets that professional settings demand.They present MBABench-2.0 as a transparent, scalable testbed for agent-built spreadsheets.Source: §6 · p. 16

A.1 Rubric Design and Applicability Annotation

• Finance experts designed the rubric to assess quality beyond numerical correctness: Accuracy accounts for 19% of its weight, with the remainder covering dimensions such as formatting, maintainability, and formula transparency.Criteria are scored as satisfied only when no mistake is found.Source: §A.1 · p. 21
• For each task, authors annotate applicable conditional criteria before grading; tasks are scored on 105–126 criteria, and all 12 categories contribute.Inapplicable criteria are withheld from the judge and their weight is redistributed within their category.Source: §A.1 · p. 21
• Final calculation accuracy is checked deterministically against the golden solution, while 19 other criteria use fixed rules and 109 are assessed by an agentic judge.Permitted differences in display, required precision, or outflow sign do not count as errors, but a different unit scale does.Source: §A.1 · p. 22
• The 129-criterion rubric combines 93 universal checks with 36 conditional checks, aiming to cover common standards and model-specific requirements across tasks.The shared rubric is designed to be comprehensive, extensible, and verifiable without writing a new rubric for each task.Source: §A.1 · pp. 21–22

A.2 Weighting of the Rubric Criteria

• Criteria receive one of four importance levels, with convex base weights; nine readily carried-over criteria receive a 0.35 discount, while category weights remain fixed.When all criteria apply, universal criteria account for 77% of total weight; the 12 ultra-important criteria contrast with 24 that are not absolutely crucial.Source: §A.1 · p. 22, §A.2 · pp. 23–24

A.3 Pass Rule and Human Baseline

• The pass rule requires correct final results and practitioner modeling conventions, while tolerating limited violations at each importance level.The paper does not report a human pass-rate baseline because results would depend on modeler experience, time, and review, and a reliable study would require substantial resources.Source: §A.3 · pp. 25–26

A.4 Evaluation Criteria

• The rubric assesses accuracy, assumptions, documentation, formula explanations, flexibility, and formatting, including whether inputs are traceable, models are adaptable, and workbook conventions are consistent.It distinguishes correct inputs, calculations, and required outputs; evaluates documentation and navigability; and checks that non-obvious formulas explain their rationale.Source: §A.4.1 · p. 26, §A.4.2 · p. 27, §A.4.3 · pp. 28–29

A.4.4 Category: Error Checks

• Error Checks assess internal consistency, visible errors and warnings, unit consistency, dedicated validation checks, and workbook hygiene such as file size and recalculation time.The category combines arithmetic consistency with checks that surface failures without manual tracing.Source: §A.4.4 · p. 30

No formula errors

• Models must contain no visible Excel errors, including common reference, value, name, and modern Excel spill or calculation errors.The criterion lists errors such as #REF!, #N/A!, #DIV/0!, #VALUE!, #NUM!, #NULL!, #NAME?, #SPILL!, and #CALC!.Source: No formula errors · p. 30

IFERROR/IFNA used only for expected errors

• IFERROR or IFNA should suppress only expected errors; blanket suppression that hides genuine errors violates the criterion.An expected case is a lookup returning #N/A in early periods before data exists.Source: IFERROR/IFNA used only for expected… · p. 31

Error-check sheet

• A dedicated error-check sheet or summary must centralize integrity tests and automatically flag failures, rather than leaving checks scattered across calculation sheets.Examples include balance, cash-tie, and roll-forward checks.Source: Error-check sheet · p. 32

Master error flag

• A cover or summary sheet should use a master error flag and clear conditional formatting to immediately show whether a check has failed.Source: Master error flag · p. 32

A.4 Evaluation Criteria

• Other criteria examine driver-based architecture, professional formatting, readable and consistent formulas, relevant outputs, workbook dangers, appropriate scope, and rounding practice.The rubric favors readable, verifiable formulas and checks that the model’s outputs serve its purpose, avoid misleading behavior, and preserve precision through calculations.Source: Driver-based, formula-linked… · p. 33, §A.4.6 · p. 33, §A.4.7 · p. 38

No errors under extreme-but-plausible inputs

• The model should remain functional under extreme-but-plausible assumption changes, which can expose hidden breaks, imbalances, errors, or nonsensical outputs.Examples include tripling useful life, shifting the valuation date by twelve months, or halving revenue.Source: No errors under extreme-but-plausible… · p. 42

A.4.12 Category: Structure

• Structure evaluates workbook architecture and sheet layout, including component selection, tab organization, fiscal-calendar fit, timeline presentation, calculation grouping, and balance construction.It also considers whether the architecture fits the model’s deal or planning orientation and how circular references are constructed.Source: §A.4.12 · p. 44

B.1.1 Procedure and Weighting

• Tasks were scored across four categories, and authoring costs limited per-level and per-type counts, so difficulty findings focus on harder levels jointly rather than comparing agents by type.Each task received a 1–6 score in Scope, General Modeling, Finance Knowledge, and Implementation in Excel; weighted scores were rounded, and the sole task reaching level 6 was reclassified as Hard.Source: B Task Annotation · p. 48, §B.1.1 · p. 48

Difficulty Categories

• Difficulty categories capture task breadth and structural complexity, quantitative modeling rigor, required finance knowledge, and Excel proficiency.Scope is not based on duration alone; finance-knowledge scoring accounts for assistance in the instructions, and Excel levels range from basic arithmetic to advanced functions.Source: §B.1.2 · p. 48, §B.1.3 · p. 49, §B.1.4 · p. 50

Exemplary Tasks by Difficulty Level

• No task was classified as Very Easy or Very Hard; the task that reached Very Hard was reclassified as Hard.DailyDetail illustrates a Hard task: an extensive debt waterfall with interdependent components, multiple scenarios, and error-prone modeling.Source: Very Easy · pp. 52–53, Very Hard · p. 53

B.2 Time Estimates

• Human modelers with approximately 3+ years of financial-modeling experience estimated task completion time, including interpretation, model building, answers, checks, and initial quality assurance.Model-building time was estimated from task-specific substeps, using the average of each substep’s estimated range.Source: §B.2 · pp. 53–54

B.3 Task Types

• The task taxonomy includes valuation models such as DCF, trading comparables, sum-of-the-parts, and NAV, as well as financial-statement models including 3-statement and working-capital models.It also includes ratio analysis, which diagnoses and benchmarks performance rather than forecasting it.Source: DCF (Discounted Cash Flow) · p. 54, Comparables (Trading Comps) · p. 54, Sum-of-the-Parts Valuation · p. 54
• Other task types cover FP&A, scenario and sensitivity analysis, debt and financing, fixed-income pricing, and personal-finance planning.Examples include budget-versus-actual analysis, debt waterfalls, bond pricing, and rent-versus-buy analysis.Source: Budget vs. Actuals Variance Analysis · p. 55, Scenario / Sensitivity Analysis · p. 55, Debt Schedule / Waterfall Modeling · p. 55

Retirement / Savings Calculator

• The Retirement / Savings Calculator projects future savings from contributions, returns, and time horizon to support long-term goals such as retirement or broader financial planning.Source: Retirement / Savings Calculator · p. 56

Business & Data Analytics

• Business & Data Analytics includes non-financial and operational analytical models.Source: Business & Data Analytics · p. 56

Non-Financial or Non-FP&A Model

• Non-Financial or Non-FP&A models analyze operational, marketing, or product activity, emphasizing data-driven insights rather than purely financial results.These models remain model-driven.Source: Non-Financial or Non-FP&A Model · p. 56

C Task Design

• Tasks are original finance-expert constructions with input workbooks, detailed instructions, assumptions, and a golden solution, and each passes at least two error-free quality-review runs.Questions have definitive answers, while the golden solutions provide ground truth for calculation accuracy rather than presentation-ready examples.Source: C Task Design · pp. 56–57

D Score Leaderboard

• Figure 9’s scores are closer across agents than pass rates, because structural and formatting criteria may be met even when an attempt fails its task.Figure 9 complements the pass-rate leaderboard in Figure 6.Source: Figure 9 · p. 58, D Score Leaderboard · p. 58

E Expert Team and Effort

• Four finance-trained experts developed the benchmark, spending approximately 2,095 person-hours on its rubric, tasks, judge, and evaluation pipeline.The team included three industry practitioners and one financial-modeling and spreadsheet-competition expert.Source: E Expert Team and Effort · p. 58

F.1 Agentic Judge Details

• The GPT-5.6 Sol agentic judge evaluates applicable criteria using workbook contents and metadata, while calculation accuracy and 19 other criteria are checked deterministically.Figure 10 reports high agreement with expert annotations; its false-positive rate is much lower than its false-negative rate.Source: Figure 10 · p. 59, §F.1 · p. 60

F.2 Per-Criterion Validation Examples

• Each of the judge-graded criteria receives one clean and one manually perturbed validation example, and the judge grades each example three times.The 109 criteria therefore yield 218 validation examples; the judging setup was refined iteratively.Source: §F.2 · p. 60

# Financial Modelling — House Standards

• House standards specify consistent formulas, colors, number formats, signs, units, and period conventions, with assumptions documented and traceable to sources or identified as judgments.Instructions govern when they conflict with the standards, and reasoned departures are allowed.Source: # Financial Modelling — House Standards · pp. 61–62
• The standards call for consolidated checks and a cover-page error flag, a linked contents page and change log, and models built for foreseeable reuse.For continued use, forecast extensions and business-unit changes should rely on added columns, inputs, and switches rather than structural rebuilds.Source: # Financial Modelling — House Standards · pp. 61–62

H Models

• The model list uses shortened names for GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash, with suffixes identifying harnesses.GPT-5.6 Sol also serves as the judge backbone.Source: Table 5 · p. 63

I Agentic Harnesses

• A harness is the deployed product surrounding a model, including its scaffold, tools, workbook handling, defaults, and workflow; comparisons do not isolate any component’s effect.The common protocol starts from the input workbook and house standards, with the rubric withheld.Source: I Agentic Harnesses · p. 63
• Each agent has one recorded attempt per task, and infrastructure failures are retried rather than recorded.The evaluation prioritizes the number of agents over repeated runs because long-horizon attempts are costly to repeat.Source: I Agentic Harnesses · p. 63
• Harnesses include GUI-controlled consumer products, code environments, Excel add-ins editing workbooks in place, and an in-house API loop with 21 spreadsheet tools.The in-house harness caps attempts at 40 model calls and recalculates formulas with LibreOffice.Source: I Agentic Harnesses · p. 64

J Qualitative Analysis — General Agent & Harness Analysis

• As tasks become harder, agents still submit structurally complete workbooks but make more mistakes; checks often test internal consistency, while complex models grow deeply nested.Harness choice also matters: Claude models in Claude for Excel match Cowork on answers but lose scaffolding and polish, while frontier coding agents match GUI counterparts on answers and score, not always pass rate.Source: J Qualitative Analysis — General Agent… · p. 65

J.1 What Agents Generally Do Well

• Agents maintain structural completeness as tasks get harder, delivering live driver-based formulas, anchored references, centralized assumptions, roll-forwards, and dedicated check rows.Blank answer sheets remain rare at every difficulty level.Source: §J.1 · p. 65
• On tasks with standard mechanics, most agents answer all or nearly all questions correctly, even on Hard tasks.On Hard task LogisticsCategorization, every agent’s stored answers match the golden solution.Source: §J.1 · p. 65
• Agents commonly provide covers explaining model purpose and methodology, descriptive tab names, and navigation that lets readers orient themselves without a walkthrough.GUI agents, Excel add-ins, and two frontier coding agents consistently produce live formula-based output summaries.Source: §J.1 · p. 65
• Workbooks usually avoid hidden or sensitive elements and maintain consistent signs and units, reducing common sources of hidden and order-of-magnitude errors when shared.Monetary blocks often carry unit tags, and scale conversions are explicit.Source: §J.1 · p. 65
• Nearly every agent includes a check sheet and master error flag, but COUNTIF roll-ups can skip error rows and report “OK” despite failed checks.Such checks can help auditors verify internal consistency, but a false all-clear can be worse for a client than no feedback.Source: §J.1 · p. 66
• Agents commonly use blue font for hardcoded values, thousands separators, consistent formats, shaded headers, and bold totals.These are basic visual guidelines for financial models.Source: §J.1 · p. 66

J.2 What Agents Generally Do Poorly

• Timelines often hardcode the requested horizon, making changes to duration, start date, or rates require manual edits; this is rare on Easy tasks but common from Medium onward.The pattern follows the model more than the harness, and models are routinely rerun with new data and conditions.Source: §J.2 · p. 66
• Workbooks often lack practical reading and print setup: freeze panes may be absent or excessive, page breaks and print areas are often missing, and narrow columns show ###.Figure 11 illustrates excessive frozen panes; Figure 12 shows values obscured by narrow columns.Source: §J.2 · p. 66, Figure 11 · p. 67, Figure 12 · p. 67
• Most “Sense Checks” compare a model with its own assumptions or trivial constraints, so they can confirm arithmetic while missing implausible valuations or cash positions.Such checks do not establish whether outputs are plausible.Source: §J.2 · p. 66
• Important inputs are rarely protected against incorrect entries: validation alerts may be disabled, validation may omit dates and rates, or no validation is added.Named ranges for global drivers are also rare outside a few agents.Source: §J.2 · pp. 66, 68
• Color conventions drift within sheets, with comparable calculations switching between green and black and weakening the legend’s usefulness for tracking cell relationships.A contradictory or inconsistent legend can be more misleading than having no legend.Source: §J.2 · p. 68, Figure 13 · p. 68
• Financial presentation conventions are inconsistently applied: negatives may use minus signs instead of parentheses, zeros may display as 0.00, and number blocks may be misaligned.Figure 14 shows negative numbers displayed with minus signs rather than parentheses.Source: §J.2 · p. 68, Figure 14 · p. 68
• From Medium-Hard tasks onward, agents often use long, nested formulas and brute-force engines; these models can be inefficient and difficult to review.One Hard-task example uses nested IF, SUMIFS, and INDEX formulas across roughly 150,000 cells; formula notes and output summaries also become less frequent, while some Code-harness answer cells show errors.Source: §J.2 · pp. 68–69

K.1.1 Fable 5.1 (Claude Code, Claude Cowork, Claude for Excel)

• In Claude Code and Claude Cowork, Fable 5.1 is narrowly the most accurate model in the field and often tests outputs against outside benchmarks while delivering well-sourced, documented workbooks.Its strengths include checking outputs against external benchmarks, citing assumptions, and explaining non-obvious formulas.Source: §K.1.1 · pp. 69–70
• Claude Code and Cowork often add sense checks against external or judgment ranges, whereas many other models only reconcile outputs with their own inputs.The ranges can still sometimes derive from the model’s own inputs.Source: §K.1.1 · pp. 69–70
• Fable often gives global drivers self-explanatory defined names and builds workbooks with a cover, linked contents, standard components, and a master error flag linked to checks.Examples include named anchor dates and tax rates, plus check flags that turn red when failures occur.Source: §K.1.1 · p. 70
• Across the harnesses, text columns or rows can be oversized while numeric or timeline columns are too narrow, leaving values displayed as ###.In one Claude Code task, text columns are 120 to 130 characters wide while the loan amount is obscured.Source: §K.1.1 · pp. 70–71
• Across harnesses, Fable sometimes labels display precision as “rounded to” even though formulas retain full precision, and it may show negative numbers with minus signs rather than parentheses.In one example, none of 12,483 schedule formulas under a “rounded to $0.01” label applies a rounding function.Source: §K.1.1 · p. 71
• Claude Code and Cowork commonly use fixed-width timelines, copied engines, or monolithic formulas, so users may need to edit ranges manually and reviewers face more complex models.A DetailedLoan timeline is fixed at 1,008 columns; a CashNiagara waterfall is copied three times, while another task uses deeply nested formulas.Source: §K.1.1 · p. 71
• Claude for Excel can produce lean engines, but its finishing is weaker, with more unexplained formulas and misuse of reserved font colors than in the other harnesses.The harness comparison also finds that Code and Cowork more often use sourced assumptions and formula notes, while Excel’s same-sheet calculations may use the green reserved for cross-sheet links.Source: §K.1.1 · pp. 71–72

K.1.2 Opus 5 (Claude Cowork, Claude for Excel)

• Opus 5 in Claude Cowork often checks headline outputs against dated outside benchmarks or judgment ranges and gives global inputs clear defined names.Examples include checking debt service and yield against external ranges and naming ten global drivers.Source: §K.1.2 · p. 72
• In Claude Cowork, timelines usually use one anchor date and a length driver, with buffer rows or columns that let users insert line items without breaking totals.One example builds a 60-quarter horizon from a single anchor.Source: §K.1.2 · p. 72
• In Claude for Excel, Opus 5 usually builds compact engines and freezes long-sheet headers and labels sensibly, but often omits hyperlinks, print areas, and explanations for non-obvious formulas.Examples include an unlinked contents list, a workbook with no print areas, and unexplained revolver and vesting-offset formulas.Source: §K.1.2 · pp. 73–74
• Check flags are unreliable in both harnesses: Cowork formulas can return #VALUE!, while Excel master flags may be static text or color and fail to signal errors.In one task, 18 check formulas return #VALUE!; in another, “ALL CHECKS PASS” appears while the debt waterfall contains #NAME? errors.Source: §K.1.2 · p. 73
• Formatting is loose across both harnesses, with cross-sheet formulas sometimes black or inconsistently green and black, and numbers often centered in Cowork.The harness comparison attributes formatting issues to both, while noting other differences in engine compactness and documentation.Source: §K.1.2 · pp. 73–74
• In Claude Cowork, number and date columns can be too narrow, obscuring summary results, reconciliation values, or dates as ###.In one task, four summary outputs are displayed as ###.Source: §K.1.2 · p. 73
• Opus 5 may append model controls, scenarios, and derived drivers below an inherited assumptions sheet’s “End Sheet” marker rather than within its declared structure.One example places case and scenario controls and an anchor date below the marker in row 121.Source: §K.1.2 · p. 73

K.2.1 GPT-6 Astra (ChatGPT Work, Codex)

• GPT-6 Astra builds polished, print-ready, well-documented workbooks with a working master error flag, but leaves key inputs unnamed and tests outputs only against the workbook itself.These patterns are reported in both harnesses.Source: §K.2.1 · p. 74
• Both harnesses commonly provide print areas and linked workbooks with source citations, formula notes, and a table of contents linking to each sheet.Examples include manual page breaks on multi-page sheets, source notes, and working hyperlinks to every sheet.Source: §K.2.1 · p. 74
• Both harnesses link a summary-sheet master error flag to check-sheet results and conditionally format it to turn red when checks fail.Examples tie the flag to failed-check counts or a formula-linked check roll-up.Source: §K.2.1 · p. 74
• ChatGPT Work formulas usually guard edge cases, while Codex applies active validation alerts and a consistent house style with blue inputs, black formulas, green links, and parenthesized negatives.Stress tests in one ChatGPT Work task found no break when duration and working-capital drivers changed.Source: §K.2.1 · p. 75
• Codex timelines roll from a central anchor date and period controls, whereas ChatGPT Work may fix the timeline length and switch off validation alerts.The Codex approach avoids formula edits when changing the horizon, but its prebuilt timelines come at the price of oversized files.Source: §K.2.1 · pp. 75–76
• Both harnesses leave global drivers unnamed and may place controls and derived inputs below inherited “End Sheet” markers; ChatGPT Work may also stack extra timelines on existing ones.The model’s inward-looking sense checks also stand out in both harnesses, while Codex’s robust timelines can produce oversized files with many formulas.Source: §K.2.1 · pp. 75–76

K.2.2 GPT-6 (ChatGPT Pro)

• GPT-6 leads on flexible timelines, honest rounding, and uniform fonts, but overbuilds, skips freeze panes, and rarely checks against outside benchmarks.The same page-break and content-below-“End Sheet” issues appear in both GPT-6 harnesses, which the paper attributes to the model family rather than the harness.Source: §K.2.2 · p. 76
• Timelines follow central date and period inputs, while a visible master pass/fail flag rolls checks onto the summary.Calculation sheets preserve full precision and identify displayed precision; only the final answer block applies ROUND.Source: §K.2.2 · pp. 76–77
• Built sheets use Arial 10 with consistent styles, and multi-page sheets receive manual page breaks.The examples include uniform styling across all built sheets and page breaks at section or period boundaries.Source: §K.2.2 · p. 77
• Inputs carry source citations, judgment calls identify an owner and date, and non-obvious modeling blocks receive methodology notes.The examples include WACC iteration, depreciation vintages, and cash-tax timing.Source: §K.2.2 · p. 77
• Brute-force grids and duplicated calculation modules make GPT-6 workbooks inefficient and difficult to review or modify.One example uses tens of thousands of reserve-grid formulas for 40 payments; another duplicates a 17,521-formula waterfall engine.Source: §K.2.2 · p. 77
• Sense checks rely on internal reconciliations, workbook history, or trivial bounds rather than outside benchmarks.In one example, MAE and MAPE are checked only against another forecast in the same workbook.Source: §K.2.2 · p. 77
• Long sheets often lack freeze panes or freeze too many rows, while some tables have erratic row heights and misaligned text.One example spans 149 columns without frozen labels or headers; another has a row height of 310 points.Source: §K.2.2 · pp. 77–78
• When extending an inherited assumptions sheet, GPT-6 can append controls and live calculations below its retained “End Sheet” marker.The added content then sits outside the sheet’s declared structure.Source: §K.2.2 · p. 77

K.2.3 GPT-5.6 Sol (ChatGPT for Excel)

• GPT-5.6 Sol builds readable, well-scaffolded workbooks with print areas and input-driven timelines, but misses several conventions and sometimes creates bloated schedules.Reported lapses include rounding labels, font colors, buffer rows, a glossary, and lean schedules.Source: §K.2.3 · p. 78
• Long narrative text is wrapped to keep sheets readable, and the model usually defines a print area on every tab.The EasyDCF example has no excessively wide built column, and long text is wrapped.Source: §K.2.3 · p. 78
• A prebuilt 30-year horizon uses central start-date and period inputs so timeline labels can roll forward without formula edits.The DebtFlow example builds the full horizon in advance.Source: §K.2.3 · p. 78
• A summary-first workbook skeleton includes a live summary, table of contents, check sheet, and change log.The summary’s values and check status link to assumptions, working-capital, and check sheets.Source: §K.2.3 · p. 78
• Sheets are sometimes labeled “rounded to $0.01” even though formulas retain full precision and do not round.In one example, none of roughly 9,900 formulas uses a rounding function.Source: §K.2.3 · p. 78
• Font colors sometimes contradict the model’s own legend, leaving cell color unreliable as a guide to formula type.Same-sheet calculations are green in three columns and black in a fourth in one example.Source: §K.2.3 · pp. 78–79
• Totals can sit directly below fixed SUM ranges, so inserting a row above a total leaves it outside the sum.The Office example had totals without buffer rows, although every other agent satisfied the criterion on that task.Source: §K.2.3 · p. 79
• Prebuilt schedules can extend far beyond the active horizon, causing repeated functions to rescan thousands of mostly blank rows.In one example, 5,200 periods were scheduled but only 180 were active.Source: §K.2.3 · p. 79

K.3.1 Grok 4.6 (Codex)

• Grok 4.6 names key inputs, freezes long sheets sensibly, and documents rounding honestly.Its examples include descriptive defined names, frozen headers and period labels, and notes distinguishing displayed precision from actual ROUND operations.Source: §K.3.1 · p. 79
• Grok 4.6 also uses cluttered formatting, hardcoded summaries, deeply nested formulas, and occasionally produces #NAME? errors.It merges cells, centers numbers, boxes calculation grids, and can create oversized rows; Bosch’s summary contained typed snapshots without formulas, while a TMobile percentile formula remained monolithic despite helper rows.Source: §K.3.1 · pp. 79–80

K.4.1 Kimi K3 (Codex)

• Kimi K3 gets workbook navigation and sheet separation right, keeps built rows compact, and documents rounding honestly.Its examples include working links to every worksheet and separate sheets for workbook functions; the only tall rows in one example were inherited instructions.Source: §K.4.1 · pp. 80–81
• Kimi K3’s summaries are static, assumptions often lack sources, and timelines and formatting conventions are inconsistently applied.It also often omits frozen headers on large sheets; one example had 14,601-row engines without a frozen header row.Source: §K.4.1 · pp. 80–81

K.5.1 Gemini 3.8 Flash (Codex)

• Gemini 3.8 Flash keeps builds proportionate and compact, uses sensible dimensions, and clearly distinguishes totals and closing balances.The SpendingPlan workbook was 488 KB for a scenario dataset of 7,500 rows.Source: §K.5.1 · p. 82
• Gemini 3.8 Flash uses rigid hardcoded timelines and static summaries, and its master error flag cannot visually signal a failed check.For example, one engine was fixed at 60 periods, while another summary had 61 constants and no formulas.Source: §K.5.1 · p. 82
• Gemini 3.8 Flash also has unguarded formulas that can fail under stress, thin user documentation, unsourced assumptions, and unfrozen, over-boxed sheets.In EasyDCF, a zero pretax-income stress caused #DIV/0! to flow into the average tax rate and valuation; long sheets also commonly lacked frozen panes and used thin borders around every cell.Source: §K.5.1 · pp. 82–83

Inputs, assumptions, and drivers

• Inputs are directly entered values, while drivers are inputs on which many calculations depend.Examples include growth or tax rates as inputs and the valuation date as a driver.Source: Inputs, assumptions, and drivers · pp. 87–88

Merged cells

• Merged cells can interfere with formulas, sorting, and references; Center Across Selection provides a similar appearance without merging.Source: Merged cells · p. 88

Named range

• A named range assigns a descriptive name, such as TaxRate, to a cell or range for use in formulas instead of cell addresses.Source: Named range · p. 88

Sense check

• A sense check compares key results with external references, such as industry benchmarks or comparable companies, to identify implausible results.Source: Sense check · p. 88

Sign convention

• A sign convention applies consistent signs, such as costs and outflows as negative and revenues and inflows as positive.Source: Sign convention · p. 88

Units and currency

• Units and currency specify the scale and currency of amounts, such as $mm for millions of U.S. dollars.Source: Units and currency · p. 88

Volatile functions

• Volatile functions, including OFFSET, INDIRECT, and NOW, recalculate whenever the workbook changes and can slow large models.Source: Volatile functions · p. 88