GoT-CQA: Graph-of-Thought Guided Compositional Reasoning for Chart Question Answering

1. Introduction

• CQA requires answers grounded in the given chart’s visual and underlying data, rather than common sense or priors.Chart parsing must recover details such as coordinates, legends, point locations, and bar heights.Source: §1 · pp. 1–2
• Chart questions can require chains of operations such as locating chart elements, estimating values, averaging, comparing, and subtracting.For example, comparing descendant counts requires locating bars, estimating values, and taking a difference.Source: §1 · p. 2
• PlotQA-D organizes questions into 74 templates across structural understanding, data retrieval, and reasoning, making a unified flexible framework desirable.Designing a separate model for each question type is described as time-consuming, expensive, and difficult to generalize to new types.Source: §1 · p. 2
• GoT-CQA transforms a chart-oriented question into a directed acyclic Graph-of-Thought whose ordered localization, numerical, and logical operations guide reasoning.Its modules parse the chart and question, perform compositional reasoning, and generate answers.Source: §1 · p. 2
• The authors present automatic composition as applicable to varied chart questions and as enhancing reasoning interpretability to a certain extent.The framework combines reasoning operations according to the question’s GoT.Source: §1 · p. 2
• GoT-CQA’s experiments on ChartQA and PlotQA-D report improvement, especially on complex reasoning questions.The paper presents the model as a first attempt to address complex reasoning in CQA.Source: §1 · p. 2

2. Methodology

• The framework assigns chart and question parsing, GoT-guided reasoning, and answer generation to three modules.The parsing module extracts chart features and generates the question’s GoT; the answering module uses the reasoning result.Source: §2 · pp. 2–3, Figure 2 · p. 3

2.1. Chart & Question Parsing Module

• Donut encodes charts as ordered visual features, while questions become DAGs of localization, numerical, and logical operators linked by prerequisite dependencies.For the descendants example, location nodes precede value-estimation nodes, which in turn precede subtraction.Source: §2.1 · p. 3

2.2. Compositional Reasoning Module

• Self-data reasoning builds Ostart; each GoT operator block fuses its textual guidance with precursor chart flows, and activated blocks pass outputs onward until a terminal result emerges.The architecture of the reasoning network varies by question; the terminal node’s output supplies evidence to the answering module.Source: §2.2 · pp. 3–4

2.3. Answering Module

• A transformer decoder consumes terminal reasoning flow Oend and generates answers that may be yes/no, chart text, or numerical-operation results.The model is trained with cross-entropy loss.Source: §2.3 · p. 4

3.1. Experimental setting

• ChartQA includes real-world charts and human-authored questions, while PlotQA-D uses programmed charts and 74 question templates; nine recent CQA systems serve as competitors.GoT-CQA uses Qwen2-7B to generate ChartQA GoTs and a template expert library for PlotQA, with a two-stage ChartQA training setup and single-stage PlotQA training.Source: §3.1 · pp. 4–5

3.2. Performance Comparison

• GoT-CQA outperforms competitors on ChartQA except for MatCha’s Augmented score, with a nearly 3.2% advantage over the best comparison on Human questions; on PlotQA-D it improves test data except overall PlotQA-D2, where MatCha leads.Across models, ChartQA Human scores are lower than Augmented scores; on PlotQA-D, scores worsen from structural to data retrieval to reasoning, and reasoning performance is below 87%.Source: §3.2 · p. 5, Table 1 · p. 5, Table 2 · p. 6

3.3. Analysis on GoT Guided Compositional Reasoning

• Removing GoT guidance lowers overall performance, while three Loc+Num+Log operators are more effective on ChartQA Human and PlotQA-D reasoning questions than one or two operators.The adaptive pattern uses more reasoning operations and time for complex questions, but fewer blocks for simple questions, avoiding excessive time overhead.Source: §3.3 · pp. 5–6
• Four self-data reasoning layers yield satisfactory results; shallow layers limit comprehensive understanding, while deeper layers risk overfitting.One-layer Loc/Num/Log blocks perform well across question types, and adding layers can hurt simple classification questions.Source: §3.3 · p. 6
• SELF-CROSS attention is preferred over pure cross- or self-attention stacking because reasoning requires both internal and interactive information mining.Source: §3.3 · p. 6

3.4. Case Study

• Case studies show GoT-CQA ahead of MatCha and UniChart on ChartQA human questions and closer to ground truth than MatCha on PlotQA-D reasoning questions.MatCha and GoT-CQA both perform satisfactorily on structural and data-retrieval cases; all models miss one ChartQA case, possibly due to visual feature extraction or self-data reasoning limitations.Source: §3.4 · pp. 6–7

• Chart-to-table systems extract chart components and underlying data, but combining this step with table QA can require time-consuming annotation and accumulate errors.The passage also notes that answering a question may not require understanding all underlying chart data.Source: §4 · p. 7
• Chart-to-text methods generate captions or summaries, but the cited approaches do not reflect high-level meanings such as data trends.Examples include controlled captioning with metadata and summarizing paragraphs that mention a chart.Source: §4 · p. 7
• Earlier CQA datasets and models often treat answering as classification over fixed vocabularies, while PlotQA-D introduced regression questions requiring numerical reasoning.Recent systems use large pre-trained vision-language models, but the passage says CQA performance remains far from human level and identifies weak reasoning as an important factor in failures.Source: §4 · pp. 7–8

5. Conclusion

• GoT-CQA’s core is a compositional reasoning module guided by a Graph-of-Thought (GoT).The paper describes GoT-CQA as a multimodal reasoning model for challenging chart question answering.Source: §5 · p. 8
• The GoT transforms a complex question into an operation sequence of simple operators, supporting an automatic compositional reasoning pattern suited to various chart-oriented questions.Source: §5 · p. 8
• Experiments on ChartQA and PlotQA-D show GoT-CQA’s effectiveness and superiority.Source: §5 · p. 8
• The authors identify exploring more reasonable operators as future work to improve GoT-CQA.Source: §5 · p. 8