PaperlayerView exact revision on arXiv

Source-linked AI summary

Faithful Chart Generation for Multimodal Deep Research: Frame-Evidence Co-Adaptation

Yuxin Yue, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Xueqi Cheng

arXiv:2610.00374v1cs.HCcs.AIcs.CLcs.GR

TL;DR

Charts make implicit quantitative claims, yet fixed visualization plans may require evidence the web retrieval process only partially supports, and post-hoc verification cannot adapt the frame. FECA iteratively adapts visual frames according to retrieved evidence. Across 100 real-world research topics, it substantially improves numerical fidelity while maintaining report and chart quality.

  • Problem

    Fixed visualization plans may require evidence that retrieved web evidence only partially covers, while post-hoc verification cannot adapt the underlying frame.

    Source: §1 · p. 2

  • Method

    FECA iteratively couples evidence acquisition with visual-frame evaluation so evidence can accept, revise, or drop a frame before chart generation.

    Source: §1 · p. 2

  • Results

    Across 100 real-world research topics, FECA substantially improves numerical fidelity while maintaining report and chart quality.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings indicate that trustworthy visual evidence generation requires iterative alignment between visual frames and available evidence rather than verification alone.

    Source: §1 · p. 2

    FECA depends on external evidence availability and reliability, LLM-based alignment and revision, and a fixed report outline and analytical intents.

    Source: §6 · p. 9

Abstract

from arXiv · show

Analytical charts in multimodal deep research encode quantitative claims, requiring every visualized value to be faithfully grounded in supporting evidence. Unlike retrieved images that mainly provide contextual information, charts require numerical fidelity: visualized values should not only match retrieved evidence quantitatively but also preserve its original meaning and scope. However, achieving such fidelity remains challenging because current systems usually construct visualization plans before knowing what quantitative evidence can actually be retrieved from the web. As a result, predefined plans may require entities, temporal ranges, or comparison dimensions that the retrieved evidence only partially supports. Existing approaches mainly address this issue through post-hoc verification after chart plans are fixed, enabling unsupported values to be identified but leaving the underlying visual frames unchanged. To address this challenge, we propose Frame-Evidence Co-Adaptation (FECA), an evidence-adaptive visual planning framework for multimodal deep research. Inspired by the bidirectional sensemaking process in Data-Frame Theory, FECA models chart generation as an iterative interaction between visual frames and retrieved evidence. Each visual frame is adaptive: the frame guides evidence acquisition, while retrieved evidence determines whether the frame should be accepted, revised, or dropped before rendering. By coupling visualization planning with evidence availability, FECA shifts chart generation from fixed-plan verification to adaptive evidence-grounded visual reasoning. Experiments on 100 real-world research topics show that FECA substantially improves numerical fidelity while preserving report quality and chart utility.

1 Introduction

Analytical charts make quantitative claims, so trustworthy multimodal research requires chart values to be grounded in evidence while preserving its meaning and scope. FECA adapts visual frames to available evidence, addressing a limitation of fixed-plan verification; experiments report improved numerical fidelity while maintaining report and chart quality.

Numerical fidelity: A missing dimension of visual trustworthiness

  • Numerical fidelity requires every generated chart value to be grounded in retrieved quantitative evidence while preserving the source’s meaning and scope.Charts function as structured visual evidence because each plotted value makes an implicit quantitative claim supporting report conclusions.Source: §1 · pp. 1–2

The limitation of fixed visualization planning

  • Fixed visual frames may require entities, temporal ranges, or comparison dimensions that retrieved evidence only partially covers, and post-hoc verification cannot adapt the frame.Verification can revise incorrect values or reject unsupported charts, but leaves the underlying visual frame unchanged.Source: §1 · p. 2

Diagnosing plan–evidence mismatches

  • 80.8% of plotted values remain unsupported by routed evidence after verification in the analyzed plan–then–check pipeline.Standard verification achieves only a limited reduction in the unsupported-value rate.Source: §1 · p. 2

The limitation of fixed visualization planning

  • More than three quarters of generated charts are removed by stricter verification, substantially reducing visual utility.The stricter verifier retains only charts with fully supported values, but cannot discover evidence-supported alternatives when a visual frame conflicts with available evidence.Source: §1 · p. 2

Our method: Frame–evidence co-adaptation

  • FECA makes each visual frame adaptive: it guides evidence acquisition, and retrieved evidence can accept, revise, or drop the frame before chart generation.FECA initializes a candidate frame for each analytical intent, evaluates evidence compatibility, and generates a chart when the frame is sufficiently supported.Source: §1 · p. 2

Experimental results

  • FECA substantially improves numerical fidelity while maintaining report and chart quality in experiments on 100 real-world research topics.The findings support iterative alignment between visual frames and available evidence rather than verification alone.Source: §1 · p. 2

2 Our Method

FECA formulates chart generation as an iterative process in which analytical intent remains fixed while visual frames adapt to available evidence. Accepted frame–evidence pairs ground chart generation, and the resulting report is recomposed around the visualizations that can be produced.

2.1 Task formulation and overview

  • FECA’s objective is to maximize report usefulness while requiring every chart’s quantitative values to be traceable to and consistent with supporting evidence.Faithfulness is a binary condition: every displayed value must be traceable to the corresponding evidence and consistent with it.Source: §2.1 · p. 3
  • Analytical intent stays fixed while its visual frame specifies a target measure, organizing dimensions, and data scope that can adapt to retrieved evidence.The frame guides evidence retrieval, and evidence determines whether that frame is accepted, revised, or dropped.Source: §2.1 · p. 3

2.2 Visual frame initialization

  • FECA derives analytical intents and candidate visual frames from report-level research before retrieving quantitative evidence for individual visualizations.The initial frames remain provisional because supporting evidence availability is unknown; they can later be revised or dropped.Source: §2.2 · pp. 3–4

2.3 Evidence-driven frame adaptation

  • At each iteration, Retrieve searches for sources matching the current frame’s measure, dimensions, and scope, while Align checks whether the evidence supports those requirements.A frame is accepted only when all required aspects are supported without unsupported quantitative assumptions.Source: §2.3 · p. 4
  • When evidence mismatches a frame, Revise can adapt the visualization to the same intent using retrieved data and prior attempts; otherwise, FECA drops it.The loop also drops a frame if no alternative is found or the iteration budget is reached without acceptance.Source: §2.3 · p. 4

2.4 Grounded visualization and report generation

  • After acceptance, the visualization planner specifies chart form and encoding from the evidence, and the renderer cannot alter its values or add unsupported ones.A supervisor checks specification conformance, execution, and readability; retries may fix implementation, but leave the accepted frame unchanged.Source: §2.4 · pp. 4–5
  • FECA recomposes the initial outline using textual sources, accepted frame–evidence pairs, and rendered visualizations, updating claims and placing each chart beside its supporting claim.The final report follows this recomposed narrative and integrates each successfully rendered visualization with its accepted evidence.Source: §2.4 · p. 5

3 Experimental Setup

The experiments use 100 open-ended research topics and assess numerical fidelity, report and chart quality, and numerical richness. They compare multimodal and general-purpose deep research systems under specified retrieval, model, and evaluation settings.

3.1 Dataset

  • The evaluation uses 100 MultimodalReportBench topics curated from expert-authored reports by Pew Research, Our World in Data, and the Open Knowledge Foundation.Topics require independent web research, organization, and visualization across quantitative trends, comparisons, and distributions.Source: §3.1 · p. 5

3.2 Evaluation metrics

  • Numerical fidelity is audited per chart value for value agreement and semantic consistency, with correctness requiring matching attributes and at most 10% relative error.Unresolved values receive a targeted open-web audit; a chart is faithful only if all numeric units pass.Source: §3.2 · p. 5
  • Report and chart quality are assessed across four report-level and four chart-level dimensions, each scored on a 1–5 scale.The dimensions include coverage and depth, visual integration, chart contribution, design, and figure–text consistency.Source: §3.2 · p. 5
  • Numerical richness is measured as raw numeric units per chart and the average number verified correct through either audit stage.Source: §3.2 · p. 5

3.3 Baselines

  • FECA is compared with three multimodal systems—MMDR, CogGen, and TVIR—and three general-purpose systems—EvoScientist, OpenAI-DR, and Perplexity-DR.The general-purpose systems can conduct open-web research, data analysis, and report synthesis but are not specifically designed for multimodal report generation.Source: §3.3 · pp. 5–6

3.4 Implementation details

  • Locally executed systems use GPT-4.1-mini for query formulation and GPT-4.1 for other LLM calls; FECA, MMDR, CogGen, and TVIR receive two retrieval rounds.OpenAI-DR and Perplexity-DR retain their native configurations; GPT-5.4 is used to extract chart units and verify them against evidence.Source: §3.4 · p. 6

4 Experimental Results

The experiments examine the limits of fixed-frame verification, FECA’s fidelity and report quality, the contributions of co-adaptation, and its efficiency. Across these evaluations, evidence-guided frame revision and iterative retrieval support more reliable chart generation while preserving report utility.

4.1 Limitations of fixed plan–then–check pipelines

  • 80.8% of values remain unsupported after practical verification; oracle filtering removes more than three quarters of charts and decreases Visual Integration.Practical verification raises the supported-value proportion from 11.1% to 19.2%; fixed frames prevent verification from finding evidence-compatible alternatives.Source: Table 1 · p. 6, §4.1 · p. 6

4.2 Evidence fidelity and report quality

  • 86.08% verified-correct rate and 12.34% Not Found rate give FECA the strongest numerical-fidelity results, exceeding TVIR and OpenAI-DR by 48.69 and 22.92 percentage points.These rates combine values verified from provided sources and through open-web search.Source: Table 2 · p. 7, §4.2 · p. 7
  • FECA leads on Coherence, Organization, Visual Integration, Chart Contribution, and Chart Self-Documentation, while delivering 11.7% more numeric units per chart and 4.35× more verified-correct units than the strongest baseline.It remains within 0.01 and 0.10 points of the best systems on Overall Report Quality and Coverage and Depth, respectively.Source: Table 3 · p. 7, §4.2 · p. 7
  • Human evaluators prefer FECA to MMDR and TVIR on numerical fidelity, find it comparable to OpenAI-DR, and record more wins than losses on visual integration and overall report quality.The blinded pairwise evaluation sampled 20 topics.Source: Table 4 · p. 7, §4.2 · p. 8

4.3 Ablation study of frame–evidence co-adaptation

  • 93.5% more charts per report and a 15.82-percentage-point verified-correct gain follow adding frame revision to Fixed Frame; full FECA leads on five of six metrics.Fixed Frame produces 41.1% more charts than Single-Pass but lowers its verified-correct rate by 7.40 percentage points.Source: Table 5 · p. 8, §4.3 · p. 8

4.4 Adaptation efficiency and frame resolution dynamics

  • 173.1% more charts per report from B=1 to B=4 costs 57.5% more tokens; Overall Report Quality largely saturates after B=2.Visual Integration continues to improve, and the second iteration yields the largest chart-count increase; the main experiments use B=4.Source: Figure 4 · p. 8, §4.4 · p. 8
  • 24.5% of frames are accepted in iteration one, adaptation accepts another 54.1%, and iterations three and four recover 20.0% more.Frames that remain unresolved within the budget are dropped.Source: Figure 5 · p. 9, §4.4 · p. 9

5 Related Work

Prior multimodal deep research systems improve reliability within fixed visual frames, while evidence-grounded chart generation usually assumes data is available before planning. FECA addresses the open-web setting by adapting frames to evidence acquired during generation.

Multimodal deep research

  • Existing multimodal deep research systems check charts at different stages but generally keep the visual frame fixed, limiting their ability to resolve unavailable-evidence mismatches.Multimodal DeepResearcher uses visual feedback, TVIR verifies data authenticity and cross-source consistency, and CogGen audits values against collected evidence.Source: §5 · p. 9
  • FECA lets visual frames adapt to retrieved evidence before chart generation, addressing the frame–evidence mismatch left unresolved by fixed-frame checks.The fixed-frame methods can improve reliability within their chosen frame but cannot resolve a mismatch when required evidence is unavailable.Source: §5 · p. 9

Evidence-grounded chart generation

  • Table-to-chart and document-to-chart methods assume their supporting data or source documents are available before visualization planning, leaving evidence acquisition outside their task scope.These settings include structured tables with visualization requests and data-bearing documents with automatic or predefined chart requests.Source: §5 · p. 9
  • Open-web report generation requires planning charts without knowing whether the needed evidence exists or can be reliably retrieved; FECA aligns frames with evidence acquired during generation.This differs from chart-generation settings where evidence is supplied beforehand.Source: §5 · p. 9

6 Conclusion

FECA reframes chart generation as iterative alignment between visual frames and retrieved evidence, allowing frames to be accepted, revised, or dropped. Experiments on 100 real-world research topics show improved numerical fidelity while maintaining report quality and usability.

6 Conclusion

  • Experiments on 100 real-world research topics show that FECA substantially improves numerical fidelity while maintaining analytical quality and usability of generated reports.FECA integrates evidence acquisition with visual-frame adaptation while preserving high-level analytical intents.Source: §6 · p. 9
  • FECA’s visual realizations can be accepted, revised, or dropped according to retrieved evidence, turning chart generation into evidence-grounded reasoning rather than fixed-plan verification.The framework iteratively interacts visual frames with supporting evidence.Source: §6 · p. 9
  • FECA depends on external evidence availability and reliability, and sparse, conflicting, or incomplete quantitative information may challenge it.LLM-based alignment and revision may also be affected by hallucinations, uncertainty, or imperfect judgments, so unsupported numerical content may occasionally pass through.Source: §6 · p. 9
  • FECA keeps report outlines and analytical intents fixed during co-adaptation, leaving joint adaptation of report plans, visual frames, and evidence as an open challenge.Future work will explore uncertainty-aware evidence reconciliation, verification beyond LLM judgments, and hierarchical co-evolution in long-horizon research agents.Source: §6 · pp. 9–10

Ethical Considerations

FECA uses publicly accessible web sources, but generated charts are not guaranteed to be objectively correct and reports require human verification, particularly in high-stakes domains.

Ethical Considerations

  • Generated charts may inherit outdated, incomplete, or biased source information, so reports require human verification rather than serving as authoritative evidence, especially in high-stakes domains.Human evaluation involved three graduate-student volunteers who were informed of the study purpose and assessed anonymized outputs.Source: Ethical Considerations · p. 10

Related papers

Loading 2610.00374v1…