Source-linked AI summary
When Literature Data Mislead Artificial Intelligence in Materials Discovery
Qian Wang, Ying Li, Ryuhei Sato, Hidemi Kato, Shin-ichi Orimo, Hao Li, Eric Jianfeng Cheng
TL;DR
Literature-derived solid-electrolyte conductivity data can contain reporting inconsistencies that are difficult to detect. The paper traces values from source articles to curated datasets and identifies recurring problems, including a cross-database example of a 100-fold conductivity error.
Problem
Literature-derived conductivity values may be difficult to interpret consistently, even for domain experts, while ambiguous entries can remain embedded in datasets.
Source: Results 97 · p. 7, Why this matters for databases and AI… · p. 8Method
The authors use solid-electrolyte curation experience to identify recurring reporting-problem categories, including text–figure inconsistency, annotation ambiguity, and unit inconsistency.
Source: Results 97 · p. 4Results
A cross-database example shows that ambiguous reporting can produce a 100-fold conductivity error.
Source: Results 97 · p. 7Takeaways & Limitations
Data accuracy is a prerequisite for reproducible science and trustworthy AI-assisted discovery.
Source: Why this matters for databases and AI… · p. 10The analysis is not an exhaustive statistical survey of the entire solid-electrolyte literature, but identifies representative and reproducible types of reporting issues.
Source: inconsistencies into downstream… · p. 14
Abstract
from arXiv · showhide
Artificial intelligence (AI) increasingly treats scientific literature as a data source for building databases, training predictive models, and guiding discovery. Yet literature-derived datasets often assume that reported experimental values are internally consistent and directly reusable. Here, we analyze this assumption using solid electrolyte (SE) conductivity data as a representative materials-science case. By tracing values from source articles to curated datasets, we identify recurrent text-figure mismatches, ambiguous axis annotations, unit inconsistencies, and missing measurement context. These discrepancies are often numerically plausible and therefore difficult to detect through routine preprocessing, but they can propagate as structured label noise during database construction and machine-learning reuse. A cross-database example shows how ambiguous reporting can create a 100-fold conductivity error. Our analysis reframes data accuracy as an infrastructure requirement for artificial-intelligence-driven discovery and motivates traceable reporting, curation, and validation practices for reusable scientific data. Keywords: AI for science; Data reliability; Scientific databases; Structured label noise; Literature-derived data; Materials informatics; Solid electrolytes
Results 97
The results examine how ambiguities in solid-electrolyte conductivity reporting complicate curation and can yield plausible database errors that evade routine preprocessing and propagate as structured label noise. They also motivate clearer reporting and traceable records to preserve data meaning for database quality and AI-assisted discovery.
Results 97
- A quoted 7.2×10^-4 S cm^-1 at 25 °C maps nearer 30 °C in the figure, while alternate y-axis interpretations yield 7.2×10^-4, 1.4×10^-4, or 2.4×10^-6 S cm^-1.Source: Results 97 · p. 5
- ln(σT) shifts from 0.93 to 5.54 when the reported 8.54×10^-3 S cm^-1 is converted to 0.854 S m^-1, matching the plot and exposing unit-label risk.Source: Results 97 · p. 6
- A “room temperature” value may omit the exact temperature or whether it was measured, interpolated, or Arrhenius-derived, although those descriptions are not experimentally equivalent.Source: Results 97 · p. 6
- Activation energies are not directly comparable when studies do not consistently define the temperature range and data points used for Arrhenius fitting.Source: Results 97 · p. 6
- 9 of 83 screened gel-SE papers (10.8%) showed text–figure inconsistency, while 3 (3.6%) showed annotation ambiguity; quoted-versus-plotted conductivity discrepancies predominated.Source: Results 97 · p. 6
- 25 issue occurrences across 16 inorganic-SE source cases comprised 13 unit inconsistencies, 7 annotation ambiguities, 4 text–figure mismatches, and 1 incomplete context; these counts are not literature prevalence.Source: Results 97 · pp. 6, 8
- A 100-fold error arose when a later database read the source article’s σT plot, with σ in S m^-1, as S cm^-1; downstream users could inherit that standardized value.The case illustrates how an interpretive decision can propagate beyond one database without remaining visible to later users.Source: Results 97 · p. 7
Why this matters for databases and AI 189
- Recurring reporting, unit, and axis conventions make these errors structured rather than random, potentially biasing learned correlations despite machine learning’s assumption of internally consistent, comparable labels.Source: Why this matters for databases and AI… · p. 8
- Numerically plausible ambiguities can evade routine preprocessing, remain embedded as structured label noise, distort trends and descriptor-property relationships, and weaken AI screening.Such entries may increase apparent dataset size while decreasing its true information quality.Source: Why this matters for databases and AI… · pp. 8–9
- Reusable experimental records need traceability from the original reported value through its physical meaning to the standardization decision, with databases retaining plotted quantity, units, conversions, measurement context, and curation decisions.Traceable records make databases more auditable, reduce repeated errors across data infrastructures, and improve inputs for AI-driven discovery.Source: Why this matters for databases and AI… · pp. 9–11
- A minimum conductivity-reporting standard should cross-check text values against figures or tables, define plotted quantities and units, specify measurement temperature and method, and report activation-energy fitting context.The paper identifies these practices as minimal conditions for literature values to remain reusable.Source: Why this matters for databases and AI… · p. 9
- Cross-database propagation shows that database construction—even by domain experts—cannot alone guarantee data quality, making accuracy a prerequisite for reproducible science and trustworthy AI-assisted discovery.Improving materials-discovery AI also requires a more accurate, standardized, reusable scientific record, not only better algorithms or larger datasets.Source: Why this matters for databases and AI… · p. 10
Methods 255
The methods compile experimental solid-electrolyte conductivity records and independently reanalyze literature reports, classifying reporting ambiguities and assessing their scope and propagation into curated datasets.
Data sources and selection 256
- The curated dataset contained 3,814 experimental solid electrolytes and more than 27,600 conductivity entries, spanning 2,900 inorganic, 528 polymer, and 386 gel electrolytes.Studies were selected when conductivity–temperature relationships were presented graphically or in tables, enabling independent reanalysis.Source: Data sources and selection 256 · pp. 11–12
Data extraction and reinterpretation 282
- Conductivity was re-extracted from figures and tables by reconstructing plotted relationships, converting temperature to 1000/T where needed, and checking ambiguous axes, units, text, and database values.For suspected cross-database inconsistencies, values were traced back to source articles; no single source was presumed authoritative.Source: Data extraction and reinterpretation 282 · p. 12
- Reporting issues were classified as text–figure inconsistency, annotation ambiguity, unit inconsistency, or incomplete measurement context.Categories covered mismatched text and plots, underspecified axes or quantities, unit discrepancies that could change values by orders of magnitude, and missing temperature or measurement details.Source: Data extraction and reinterpretation 282 · pp. 12–13
single class. 313
- Statistics for gel polymer electrolytes were calculated at the paper level from 83 publications, whereas inorganic solid-electrolyte statistics were calculated by issue occurrence: 16 source cases corresponded to 25 occurrences.Where ambiguity could not be uniquely resolved, cases were conservatively assigned.Source: single class. 313 · p. 13
inconsistencies into downstream datasets. 328
- The analysis identifies representative, reproducible ambiguity types rather than exhaustively surveying solid-electrolyte literature, and interprets discrepancies as features of reporting practices, not isolated errors.Graph interpretation depends on figure resolution, reporting clarity, and expert judgment; curated databases may also reflect prior literature inconsistencies.Source: inconsistencies into downstream… · p. 14
of current reporting practices rather than isolated errors. 338
- Source papers for representative cases, issue classifications used for Fig. 2D, and DigBat statistics used for Fig. S1 are provided in the supplementary tables and materials.The source papers are listed in Supplementary Tables S1 and S2.Source: of current reporting practices rather… · p. 14
Supplementary Information 416
The supplementary information catalogs solid-electrolyte literature cases with reporting issues and outlines the scope and temporal growth of DigBat data categories.
Supplementary Information
- Tables S1 and S2 catalogue gel and inorganic solid-electrolyte papers with reporting issues.The listed issue categories include annotation ambiguity, unit inconsistency, text–figure inconsistency, and incomplete measurement context.Source: Table S1 · p. 18, Table S2 · p. 19
- The cited listing identifies I3, I5, I7, and I16 as combining annotation ambiguity with unit inconsistency, while I4 and I6 are listed with unit inconsistency.I11 is instead listed with annotation ambiguity and text–figure inconsistency.Source: Supplementary Information 416 · p. 19
- Other cases combine text–figure inconsistency with unit inconsistency or incomplete measurement context.I8 and I10 list text–figure and unit inconsistencies, while I13 lists text–figure inconsistency and incomplete measurement context.Source: Supplementary Information 416 · p. 19
- DigBat statistics track the total number of materials over time across inorganic, polymer, gel-polymer, and computational categories.The figure description says its inset summarizes the unique DOI count for each category; the statistics are derived from DigBat.Source: Supplementary Information 416 · p. 20