Source-linked AI summary
SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning
Panjing He, Mingyue Cheng, Yucong Luo, Li Li, Xiaohan Zhang
TL;DR
Automated spreadsheet reasoning is limited by models’ difficulty representing cross-table dependencies, spatial layouts, and inter-sheet relationships. SheetCompass addresses this with hierarchical graphs, dual-level memory, and multi-agent reasoning, outperforming baselines across evaluated benchmarks.
Problem
LLMs struggle to preserve spreadsheets’ intra-table topologies and inter-table dependencies, limiting high-fidelity automation of complex workbooks.
Method
SheetCompass combines hierarchical graphs for spreadsheet structure with dual-level memory and a coordinated multi-agent workflow.
Results
SheetCompass consistently outperforms baseline methods across all evaluated benchmarks, reaching 63.2% SCB pass@1, 18.3% SB hard restriction, and 43.5% SheetRM pass@1 with GPT-4.
Takeaways & Limitations
Experiments indicate that structural graphs, multi-agent execution, and memory retention are central contributors to SheetCompass’s spreadsheet reasoning performance.
Abstract
from arXiv · showhide
Spreadsheets are widely used to organize, analyze, and manipulate semi-structured data, yet automated spreadsheet reasoning remains challenging for large language models (LLMs). Real-world workbooks often contain implicit cross-table associations, fine-grained column dependencies, and complex spatial layouts. Existing methods typically flatten these multidimensional structures into sequential strings, losing important intra-sheet boundaries and inter-sheet semantics. Consequently, LLMs cannot exploit the global spatial context that human experts naturally use when inspecting spreadsheets. We propose SheetCompass, a graph-guided and memory-driven agentic framework for spreadsheet reasoning and automation. SheetCompass explicitly models structural relationships within and across worksheets while maintaining task-relevant information in memory, enabling agents to reason more effectively over complex workbooks.
1 Introduction
Spreadsheet automation is important but difficult because flattening complex workbooks into linear text loses intra-sheet topology and inter-sheet dependencies that support human spatial reasoning. SheetCompass addresses this gap with hierarchical graphs and coordinated multi-agent reasoning to preserve structural relationships and enable high-fidelity task processing.
- Motivation: Spreadsheets encode business logic and domain expertise, while heterogeneous data and repetitive manipulation make automation a critical industrial productivity bottleneck.LLMs are presented as a robust technical pathway for automated spreadsheet processing.
- Problem: LLM performance declines sharply on complex spreadsheets because prevailing methods flatten multidimensional layouts into sequential Markdown or JSON representations.Although textual content is preserved, the flattening paradigm loses intra-sheet topology and other structural information.
- Problem: Complex spreadsheets contain intricate intra-table topologies and inter-table dependencies that linear sequences obscure, whereas human experts use multidimensional layout perception to extract latent semantics.Human visual scanning reconstructs spatial topology and creates implicit alignment across distant cells, while models rely on statistical structural approximation in linear sequences.
- Approach: SheetCompass restores structural dependencies by recasting neighborhoods into a hierarchical graph with table-level and column-level edges.These hierarchical edges preserve intra-sheet spatial layouts and inter-sheet semantic structures, supporting high-fidelity processing through a multi-agent workflow.
- Contributions: The framework combines structural space perception with multi-agent reasoning, a hierarchical contextual graph, dual-level memory, and a coordinated multiagent workflow.These components are described as addressing structural information loss in flat sequences, preserving spatial layouts and semantic linkages, and guaranteeing high-fidelity task execution.
2 Related Work
Related work progresses from program-synthesis methods for deterministic spreadsheet automation to LLM-based reasoning and agentic frameworks. Recent research further explores multi-agent systems that decompose complex objectives through structured roles and collaboration.
- Spreadsheet automation: Program-synthesis systems such as FlashFill search domain-specific languages for user-specified spreadsheet programs, achieving high precision on deterministic tasks.These methods face exponentially expanding DSL search spaces as constraints increase, creating a severe curse of dimensionality.
- LLM and agentic reasoning: LLM reasoning research advanced from individual-model prompting, including Chain-of-Thought, toward ReAct agents that alternate internal reasoning with external tool execution.The passage characterizes ReAct as grounding multi-step reasoning in empirical action and enabling dynamic environment interaction.
- Multi-agent systems: Multi-agent systems address monolithic-model limitations by decomposing global objectives into orchestrated subtasks using structured roles and granular labor division.Frameworks such as MetaGPT and ChatDev leverage inter-agent debate or collaboration to improve reliability and rectify model biases.
3 Methods
SheetCompass formalizes spreadsheet automation as generating target table sequences from spreadsheet collections and user queries, then addresses structural loss through hierarchical graphs, dual memories, and a coordinated multi-agent workflow. Its method reconstructs spatial and cross-sheet dependencies, preserves task history and expertise, and constrains execution through graph-grounded programming and state validation.
- Problem Formulation: Spreadsheet automation is modeled over collections of tables and user-issued instructions, with the objective of generating the ground-truth target table sequence.The dataset formalization defines each instance as (T, q, A*) and models a generated table sequence conditioned on T and q.
- Hierarchical Relation Graph: SheetCompass reconstructs lost spreadsheet structure by representing multi-level components as a hierarchical graph of table and column nodes.Column nodes pair header names with representative raw-data samples, avoiding dense cell-level modeling while preserving ownership and structural dependencies.
- Hierarchical Relation Graph: Related columns are connected using transformer representations, cosine spatial-semantic similarity, and a confidence-weighted threshold that filters low-quality links.A logical edge is drawn only when the combined score exceeds the confidence threshold, enabling discovery of relationships across distant spreadsheet regions and sheets.
- Dual Memory: SheetCompass combines permanent expert knowledge memory with task-specific reasoning experience memory to support iterative feedback and multi-step execution.Expert memory stores domain and tool-use knowledge, while reasoning memory records graph movements, sandbox errors, and checker mismatches for subsequent prompts.
- Multi-Agent Workflow: The multi-agent workflow assigns exploration, code generation, and validation to the explorer, programmer, and reflector, respectively.The explorer decomposes instructions into atomic steps and locates targets; the programmer anchors variables to verified graph nodes in a sandbox; the reflector checks post-execution metadata against instruction-derived requirements.
4 Experiments
Experiments evaluate SheetCompass on complex spreadsheet benchmarks using task-specific metrics, representative baselines, and GPT-family backbones. Results show robust gains from hierarchical graphs, dual-level memory, iterative reasoning, and verification across benchmark and configuration analyses.
- Experimental Setup: Experiments use SCB, SB, and SheetRM, with SB providing three independent test cases per task to assess reasoning robustness and accuracy.SCB and SheetRM use exec@1 and pass@1, while SB uses soft and hard restriction metrics.
- Experimental Setup: Baselines cover static generation and dynamic interaction, while GPT-5 and cost-efficient GPT-4o-mini test performance across model capacities.The dynamic frameworks use plan-action-observation cycles to adjust actions from environment feedback.
- Main Results: 63.2% pass@1 on SCB, 18.3% hard restriction on SB, and 43.5% pass@1 on SheetRM under GPT-4 surpass the strongest baselines.SheetCompass consistently outperforms all baselines across benchmarks, with a larger performance edge under GPT-5.
- Ablation Studies: Removing the hierarchical graph lowers SCB pass@1 by 14.9%, SheetRM pass@1 by 10.9%, and SB hard restriction by 7.5%.Removing structural edges reduces SheetRM exec@1 from 94.2% to 88.7%, indicating that table-column topology preserves layout information.
- Ablation Studies: Disabling expert knowledge reduces SB hard restriction from 22.0% to 17.3%, while removing reasoning experience drops SCB pass@1 from 71.3% to 68.9%.Removing heuristic exploration lowers SCB pass@1 to 64.8%, and removing the reflector lowers SheetRM pass@1 to 45.1%.
- Analysis: Increasing reasoning cycles from 1 to 2 raises SCB and SheetRM accuracy to 71.9% and 54.7%, while optimal graph thresholds are α=0.85 and dataset-specific λ values.SCB peaks at 71.3% with λ=0.6, whereas SB peaks at 26.1% with λ=0.5; semantic edges rise from 18.7% to 29.9% in multi-table settings.
5 Conclusion
SheetCompass addresses the structural and semantic gap in automated spreadsheet reasoning by representing spreadsheets as hierarchical graphs rather than flat text sequences. Its multi-agent workflow coordinates explorer, programmer, and reflector roles within an execution sandbox to emulate human expertise.
- SheetCompass is designed to bridge the structural and semantic gap in automated spreadsheet reasoning.
- The framework uses a hierarchical graph instead of flat text sequences to capture spreadsheets’ intrinsic topological hierarchies.
- Its multi-agent workflow coordinates specialized explorer, programmer, and reflector roles within a robust execution sandbox.
A Experimental Setup · A.1 Implementation Details
SheetCompass uses a Cognitive Ensemble built on gpt-5 and gpt-4o-mini, with deterministic decoding settings for complex logical synthesis. Its contextual graph maps heterogeneous raw features into a shared 1024-dimensional latent space using BGE-M3.
- A.1 Implementation Details: A.1 Implementation Details: SheetCompass uses gpt-5 as a core reasoning engine within the Cognitive Ensemble.
- A.1 Implementation Details: A.1 Implementation Details: SheetCompass uses gpt-4o-mini as another core reasoning engine within the Cognitive Ensemble.
- A.1 Implementation Details: A.1 Implementation Details: The implementation maintains temperature at 0.2 to support deterministic and stable output.
- A.1 Implementation Details: A.1 Implementation Details: The implementation sets max tokens to 4096 for complex logical synthesis.
- A.1 Implementation Details: A.1 Implementation Details: BGE-M3 serves as the pre-trained transformer Φ in the contextual graph’s mathematical framework.
- A.1 Implementation Details: A.1 Implementation Details: Transformer Φ maps heterogeneous raw features into a 1024-dimensional latent space for unified measurement.
A.2 Datasets
The evaluation uses three specialized benchmarks—SheetCopilotBench, SpreadsheetBench, and SheetRM—to assess agent capabilities over complex tabular structures. These benchmarks cover cross-sheet spreadsheet manipulation, irregular layouts, and long-horizon reasoning with ambiguous requirements.
- Benchmark Composition: The evaluation comprises SheetCopilotBench (SCB), SpreadsheetBench (SB), and SheetRM, three benchmarks for complex tabular structures.The passages identify these as specialized benchmarks designed to assess agent capabilities over complex tabular structures.
- SheetCopilotBench (SCB): 221 samples in SCB cover real-world software-control tasks involving cross-sheet dependencies, formulas, pivot tables, and visualization charts.SCB is derived from representative real-world software control tasks and focuses on advanced spreadsheet manipulations.
- SheetRM: 180 multi-category tasks in SheetRM target irregular layouts, long-horizon multi-step reasoning, and ambiguous textual requirements requiring iterative planning and reflection.The dataset is curated from professional spreadsheet software examination banks to evaluate resilience against practical operational challenges.
A.3 Evaluation Metrics
SheetCompass is evaluated with metrics tailored to benchmark-specific verification paradigms. SheetRM and SCB use immediate execution indicators alongside Soft and Hard Restriction measures to distinguish partial logic capture, complete reliability, and genuine spatial-layout reasoning.
- Benchmark-specific metrics: SheetRM and SCB are evaluated with Pass@1 and Exec@1, which quantify immediate functional success in discrete software-control workflows.These metrics report the percentage of tasks that satisfy the benchmark’s immediate execution criteria.
- Restriction metrics: Soft Restriction grants partial structural credit proportional to the success rate of individual test cases.It measures the agent’s foundational logic-capturing capability across the input-output tests associated with each instruction.
- Restriction metrics: Hard Restriction awards a positive score only when all test cases for an instruction execute flawlessly.This metric assesses absolute operational reliability and functional robustness.
- Evaluation rationale: Juxtaposing execution indicators with multi-case restrictions helps separate simple code memorization from genuine spatial-layout reasoning.The indicator function returns 1 when its condition is satisfied and 0 otherwise.
B Prompt Specifications
The prompt specifications define specialized agents for semantic spreadsheet-graph analysis, graph exploration, code generation, and runtime-based verification. Together, these prompts require structured reasoning over spreadsheet context, tool-augmented execution, and JSON-formatted outputs where specified.
- B.1 Prompt for Semantic Alignment: The semantic-alignment prompt profiles each spreadsheet node by functional role within the workbook hierarchy.It frames the task as semantic deconstruction and dependency scoring between spreadsheet nodes.
- B.1 Prompt for Semantic Alignment: The semantic-alignment prompt identifies potential links such as derive or align between nodes.Relationship confidence is represented as a probability from 0.0 to 1.0 using the spreadsheet context.
- B.2 Prompt for the Explorer Agent: The explorer-agent prompt decomposes high-level instructions into ordered, atomic reasoning steps before exploring the spreadsheet graph.Its inputs include the instruction, current step, seed ids, current subgraph, history, and turn and node budgets.
- B.2 Prompt for the Explorer Agent: The explorer agent uses topological exploration to navigate the graph from supplied seeds and the current subgraph.The prompt specifies specialized tool invocation and requires JSON-only output.
- B.3 Prompt for Programmer Agent: The programmer-agent prompt translates operational instructions and structural tabular context into Python code through a multi-turn, tool-augmented reasoning loop.It maps semantic entities and anchoring coordinates from a shared structural subgraph to execution logic.
- B.4 Prompt for Reflector Agent: The reflector-agent prompt verifies generated code against target instructions using physical runtime execution signals and workbook telemetry.Its evaluation cross-references before-and-after statistics and diffs to assess structural modifications.
C Expert Knowlege Memory … GenAI Usage Disclosure
SheetCompass uses a tiered domain knowledge base and dual-backend tool-use knowledge to provide spreadsheet-task constraints, safe execution, and task-specific capabilities. The paper also discloses its use of Codex and ChatGPT during implementation and writing, while assigning final responsibility to the authors.
- C.1 Domain Knowledge: The domain knowledge base has three tiers: static rules, proactive hints, and evolutionary failure patterns.These tiers provide logical constraints and semantic priors across diverse spreadsheet tasks.
- C.1 Domain Knowledge: Static rules and proactive hints form the foundational defense for operational safety and logical alignment.The passage identifies this combination as the first line of defense within the knowledge framework.
- C.2 Tool-Use Knowledge: The execution framework separates tools into a script-control backend and a host-application backend.The script backend uses a high-level programming language, whereas the host backend is driven by native spreadsheet software.
- C.2 Tool-Use Knowledge: Both backends support always-on loading or keyword-triggered activation according to operational requirements.The activation mechanisms are integrated into the system for backend use during execution.
- C.2 Tool-Use Knowledge: The script-control backend targets low-latency, stateless manipulation of raw file structures and requires schema-based coordinate anchoring before writes.This prevents accidental data corruption or full-table overwrites when target tables do not begin at the origin cell.
- C.2 Tool-Use Knowledge: The host-application backend handles complex tasks that depend heavily on the internal spreadsheet engine, including native pivot table creation.The supplied passage identifies native spreadsheet-software capabilities as the focus of this backend.
- GenAI Usage Disclosure: Codex assisted with baseline evaluation scripts and specific framework implementation portions, while ChatGPT improved grammatical correctness and text flow.ChatGPT was used exclusively during writing, and the disclosure follows the ACM Policy on the Use of AI.
- GenAI Usage Disclosure: The authors retain responsibility for the final phrasing, technical accuracy, and scientific arguments.This statement appears in the paper’s Generative AI usage disclosure.