Source-linked AI summary
ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoning
Jasin Cekinmez, Omid Ghahroodi, Saad Fowad Chandle, Dhiman Gupta, Ehsaneddin Asgari
TL;DR
Biographical reasoning remains vulnerable to hallucination, English-centric bias, and gaps in popularity-aware coverage. ADAM combines a multilingual biographical database, Bloom-based evaluation, and retrieval; retrieval improves performance, especially for open-source models, while face images yield smaller and less consistent gains.
Problem
Biographical reasoning faces hallucinations, weak popularity awareness, and English-centric coverage, limiting reliable information retrieval across languages and cultures.
Source: §1 · pp. 1–2Method
ADAM combines a multilingual biographical database, Bloom-based evaluation across six reasoning levels, and popularity-aware retrieval-augmented generation.
Source: §1 · p. 2Results
Retrieval consistently boosts performance, especially for smaller open-source models, while face-image conditioning gives smaller and less consistent improvements.
Source: §5 · p. 9Takeaways & Limitations
The authors conclude that retrieval is essential for bridging open- and closed-source performance gaps, while fairness and multimodal grounding need further attention.
Source: §5 · p. 9The authors call for better multimodal fusion, fairness-aware evaluation, and broader coverage of less-documented populations.
Source: §5 · p. 9
Abstract
from arXiv · showhide
We introduce ADAM (A Diverse Archive of Mankind), a framework for evaluating and improving multimodal large language models (MLLMs) in biographical reasoning. To the best of our knowledge, this is the first work to systematically examine LLM capabilities in biography, a critical yet underexplored dimension of factual knowledge. At its core, AdamDB is a multilingual and multimodal dataset covering over 4 million individuals across geography, time, and profession, while AdamBench provides cognitively structured evaluations based on Bloom's taxonomy, spanning six reasoning levels in both English and native languages. To address hallucinations, particularly for lesser-known individuals, we propose AdamRAG, a retrieval-augmented generation system tailored to biographical contexts. Experiments show that AdamRAG substantially improves open-source models and modestly benefits closed-source ones, with the largest gains on lower-order reasoning. Popularity strongly mediates accuracy, and multimodal input via face images offers smaller, less consistent improvements than retrieval. ADAM establishes the first benchmark and framework for cognitively, culturally, and multimodally grounded biographical evaluation, advancing the development of multilingual, accurate, and hallucination-resistant MLLMs.
1 INTRODUCTION
ADAM addresses factuality and coverage gaps in biographical reasoning with a multilingual resource, retrieval, and cognitively structured evaluation. Its design targets cross-language access and popularity-aware retrieval while assessing reasoning from recall to creative synthesis.
1 INTRODUCTION
- Biographical reasoning demands factual accuracy, yet LLMs hallucinate and multimodal models struggle, alongside gaps in popularity awareness and English-language coverage.These challenges make reliable biographical information across languages and cultures difficult to retrieve.Source: §1 · p. 1
- Existing biographical systems are typically monolingual, manually curated, and limited in scale, leaving temporal, cultural, and linguistic gaps.The passage also notes that no system is specifically engineered for biography’s complexities.Source: §1 · pp. 1–2
- ADAM introduces a framework for biographical reasoning with more than four million records, nearly 600 native languages, cross-language search, and popularity-weighted retrieval.AdamDB is automatically constructed through a WikiDB-to-RAG pipeline, and popularity weighting uses Wikipedia engagement metrics.Source: §1 · p. 2
- AdamBench evaluates biographical reasoning across six Bloom’s taxonomy levels, from factual recall to creative synthesis.The framework also integrates multilingual retrieval, popularity awareness, and cognitive evaluation.Source: §1 · p. 2
2 RELATED WORKS
Earlier biographical datasets serve varied tasks and populations, with coverage and structure differing across collections. ADAM is reported to exceed prior datasets in record count, supported languages, and country coverage.
2 RELATED WORKS
- Biographical datasets support tasks including summarization, named entity recognition, information extraction, and text generation, with accuracy, coverage, and structure varying by source.Sources include encyclopedias, web-scraped data, news, social media, and synthetic profiles.Source: §2 · p. 2
- Prior resources range from English Wikipedia generation and occupation-focused biographies to curated notable-person collections and person-centric timelines.The cited examples include WikiBio, BIOS, Pantheon, BiographySampo, and EventKG.Source: §2 · p. 2
- The ADAM framework combines a multilingual, multimodal biographical knowledge base, Bloom-based evaluation, and retrieval-augmented generation.AdamBench spans all six reasoning levels, while AdamRAG is designed to strengthen factual grounding and mitigate hallucinations.Source: Figure 1 · p. 3
- ADAM significantly outperforms previous datasets in record count, supported languages, and country coverage.Table 1 compares ADAM with earlier datasets on these dimensions.Source: §2 · p. 3, Table 1 · p. 3
3 APPROACH
ADAM combines AdamDB, a multilingual biographical resource, AdamBench’s cognitively structured evaluation, and AdamRAG’s retrieval grounding. Its approach covers database construction and sampling, text- and image-based retrieval, and evaluation across languages, modalities, popularity, and reasoning levels.
3.1 OVERVIEW
- ADAM integrates a biographical database, Bloom-structured benchmark, and retrieval system to evaluate and ground multilingual, multimodal biographical reasoning.AdamDB provides factual resources, AdamBench assesses cognitive skills, and AdamRAG supplies retrieved context to models.Source: §3.1 · pp. 3–4, §3.2 · p. 4
3.2 DATASET
- AdamDB contains approximately four million individuals across nearly 600 languages, built from filtered WikiDBS records aligned with Wikidata identifiers.The database spans all continents and over 200 countries; retained records require biography, birth date, nationality, and birthplace.Source: §3.2 · p. 4
- AdamBench uses multiple-choice questions in English and subjects’ native languages, spanning six Bloom levels and including image-based items.Questions are generated from selected individuals’ multilingual names and Wikipedia summaries.Source: §3.2 · pp. 4, 6
- Benchmark sampling groups people by country and popularity, then selects representatives from biography and birth-date clusters to diversify the approximately 1,650-person set.The selection aims to cover nationality, historical period, profession, and notability.Source: §3.2 · p. 6
3.3 ADAMRAG
- AdamRAG retrieves structured biographical facts from AdamDB and adds them to the prompt before generation, targeting sparse, ambiguous, and non-English cases.The system is intended to support lesser-known people, disambiguation, and multilingual aliases.Source: §3.3 · p. 6
- Text retrieval tries exact matches, then uses LaBSE similarity and nationality and birth-date filters to select a candidate.The final selection uses cosine similarity between biography embeddings and query context.Source: §3.3 · p. 7
- Face queries retrieve similar AdamDB entries, filter them by nationality and birth date, and return up to five candidates.Two verified images per person were crawled when Wikipedia photos were absent.Source: §3.3 · p. 7
3.4 EVALUATION FRAMEWORKS AND METRICS
- Evaluation uses AdamBench across cognitive levels, popularity, language, and modality, comparing retrieval-augmented responses with zero-shot prompting.The evaluation covers open-source and proprietary models under multilingual and multimodal conditions.Source: §3.3 · p. 7, §3.4 · p. 7
4 RESULTS
Across cognitive levels, closed-source models generally outperform open-source baselines, while retrieval produces substantial gains for open-source models and narrows the gap. Accuracy also varies with popularity and language, while face-image input yields mixed and comparatively smaller effects.
4.1 OVERALL TRENDS ACROSS COGNITIVE LEVELS
- Remembering and Understanding have the highest accuracies, while Evaluating and Creating expose greater weaknesses; open-source models often fall below 60% without retrieval.Closed-source models reach above 85–95% in most conditions, and the gap widens at higher cognitive levels.Source: §4.1 · p. 7
4.2 MODEL COMPARISONS
- Gemini Flash 2.5 often exceeds 95% with retrieval, while retrieval lifts weak zero-shot open-source performance and narrows the gap with closed-source systems.Gemini leads factual recall; GPT-4 is more balanced on higher-order reasoning, especially with multimodal input.Source: §4.2 · pp. 7–8
4.3 EFFECT OF RETRIEVAL-AUGMENTED GENERATION (RAG)
- Qwen2.5-7b rises from below 40% to well above 70% on mid- and high-popularity individuals with retrieval; Gemma3-12b-it exceeds 80% in Applying and Analyzing.Closed-source models also improve with retrieval, with most scores above 95%.Source: §4.3 · p. 8
4.4 POPULARITY EFFECTS
- GPT-4 zero-shot accuracy rises from approximately 65% on low-popularity people to above 90% on highly popular people, and retrieval reduces but does not eliminate the disparity.Popularity effects are also pronounced for open-source models, including Qwen2.5-7b.Source: §4.4 · p. 8
4.5 LANGUAGE EFFECTS
- Original-language questions have modest zero-shot advantages, especially for closed-source models’ Remembering and Understanding, but the difference largely vanishes with retrieval.The passage attributes the zero-shot pattern to cultural and linguistic grounding and says retrieval compensates for linguistic coverage gaps.Source: §4.5 · p. 8
4.6 MULTIMODALITY AND IMAGE INPUT
- Face-image input has mixed effects: GPT-4 declines on zero-shot Remembering, open-source gains are modest and inconsistent, and retrieval has a much larger impact.Gemini Flash 2.5 remains strong with or without image conditioning.Source: §4.6 · p. 8
5 DISCUSSION AND CONCLUSIONS
ADAM combines AdamDB, AdamBench, and AdamRAG to evaluate and improve biographical reasoning across models, languages, and modalities. The findings highlight popularity-related disparities, retrieval gains, and less consistent benefits from face images.
5 DISCUSSION AND CONCLUSIONS
- Retrieval boosted performance across models, especially smaller open-source models, and mitigated disparities across languages and popularity levels.Closed-source models initially outperformed open-source baselines, but retrieval substantially narrowed the gap.Source: §5 · p. 9
- All models performed better on widely known individuals than on less popular ones, raising fairness and inclusivity concerns.The authors associate this pattern with heavy reliance on pretraining exposure.Source: §5 · p. 9
- Face-image conditioning yielded smaller, less consistent improvements than retrieval, while Evaluating and Creating remained the most challenging tasks.Visual cues added limited value especially in lower-order tasks; the authors identify persistent abstraction gaps in higher-order reasoning.Source: §5 · p. 9
- The authors call for more principled multimodal fusion, fairness-aware evaluation, and extension to less-documented populations.They frame these directions as future work toward more accurate, culturally sensitive, and hallucination-resistant MLLMs.Source: §5 · p. 9
ETHICS STATEMENT
The authors report no human-subject experiments or use of sensitive personal data, and state that biographical information came from public sources. They also describe how they handled image rights and used LLMs as auxiliary tools.
ETHICS STATEMENT
- The work involved no human-subject experiments or sensitive personal data; biographical information came from public sources, primarily Wikipedia and Wikidata.The authors state that processing complied with the sources' respective licenses.Source: ETHICS STATEMENT · p. 9
- For multimodal content, the authors retained public URLs to original image hosts rather than redistributing copyrighted images.They state this approach preserved copyright and attribution.Source: ETHICS STATEMENT · p. 9
- LLMs were used for text polishing and literature-review retrieval and discovery; the authors say scientific content and results were validated by them.The authors characterize the tools' roles as supportive and disclose them for transparency.Source: ETHICS STATEMENT · p. 9
A ADDITIONAL DATASET STATISTICS
The appendix identifies displays for AdamDB's geographic distribution and language coverage, without specifying their underlying values in these passages.
A ADDITIONAL DATASET STATISTICS
- Figure 2 presents the distribution of AdamDB individuals by continent.The passage does not provide continent-level counts or proportions.Source: Figure 2 · p. 11
- Table 3 presents AdamDB language coverage statistics.The passage does not provide the table's values.Source: Table 3 · p. 11
- Figure 3 presents the top 10 countries by AdamDB record count.The passage does not state the counts or identify a leading country.Source: Figure 3 · p. 12