PaperlayerView exact revision on arXiv

Source-linked AI summary

WildSEEK: Evaluating Language Models for Information-Seeking

Tanise Ceron, Joachim Baumann, Elisa Bassignana, Berat Cabuk, Dirk Hovy, Debora Nozza

arXiv:2608.30683v1cs.CLcs.CY

TL;DR

Existing evaluations often rely on topic-specific or synthetic settings, leaving the complexity and risks of real-world information-seeking insufficiently captured. The paper introduces WILDSEEK and an evaluation framework, finding that analytical queries more often elicit unsafe and unfair responses and that web search does not consistently improve performance. These resources provide an empirical basis for monitoring LLM reliability, fairness, and safety in information access.

  • Problem

    Existing evaluations are often topic-specific or synthetic, limiting coverage of complex real-world information-seeking queries and risks in model responses.

  • Method

    The paper manually annotates 3,077 natural human–LLM information-seeking queries, trains classifiers for large-scale analysis, and evaluates responses for reliability, fairness, and safety.

  • Results

    Analytical queries yield more unsafe and unfair responses than factoid queries, and integrated web search does not consistently improve model performance across three evaluated LLMs.

  • Takeaways & Limitations

    WILDSEEK and its framework provide an empirical foundation for evaluating LLM behavior in realistic information-seeking scenarios beyond simple factual correctness.

  • Takeaways & Limitations

    The dataset is static, does not include emerging topics after collection, and cannot exhaustively represent all possible user interactions.

Abstract

from arXiv · show

Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existing evaluations, however, are often topic-specific or synthetic, limiting their ability to capture the complexity of "in the wild" information-seeking queries and the risks present in model responses. To address this gap, we introduce WildSEEK, a manually annotated dataset of 3k information-seeking queries from real user interactions, and an evaluation framework for LLM-generated responses. WildSEEK includes annotations for risk-sensitive domains (e.g. health and financial information), and distinguishes factoid queries from analytical queries which seek responses beyond facts. We train classifiers on WildSEEK to analyze more than 1.8M realistic user queries. We find that over a third of information-seeking queries are high-risk and more often analytical. Our findings show that LLM responses fail more often in four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations -- with failure rates being mostly higher for analytical queries. By providing methods to monitor the reliability, safety, and fairness of LLM behavior, our dataset and evaluation framework offer an empirical foundation for the broader question of how these systems should behave as they take on a growing role in information access.

1 Introduction

WILDSEEK addresses the need to evaluate LLMs as increasingly important intermediaries for complex, risk-sensitive information seeking. It combines a real-query dataset with a framework for assessing response reliability, fairness, and safety.

  • LLMs increasingly mediate information access by summarizing, reorganizing, and reconstructing complex information for users.
  • WILDSEEK contains 3,077 manually labeled information-seeking queries from natural human–LLM interactions, annotated for intent, risk-sensitive domain, and open-endedness.
  • The paper distinguishes factoid queries from analytical queries requiring interpretation, synthesis, or subjective judgment, then scales analysis with classifiers.
  • 40% of user–LLM turns are information-seeking on average, with more than a third involving risk-sensitive domains and 60% having analytical properties.
  • Across evaluated LLMs, analytical queries produce more unsafe and unfair responses, while web search does not consistently improve performance; common failures include sycophancy and overreliance.
  • The evaluation framework assesses LLM-generated responses along reliability, fairness, and safety dimensions.

2 Related Work

Prior work defines information seeking and classifies search behavior, but existing taxonomies do not fully capture the broad range of user tasks and query purposes in LLM interactions. WILDSEEK focuses on information seeking from both user and model-response perspectives.

  • Information seeking is described as a purposeful search for information and a fundamental skill in an information society.
  • Traditional web-search taxonomies distinguish navigational, informational, and transactional queries by user intent.
  • Research on LLM interactions shows use extending beyond traditional NLP tasks to advice, planning, and analysis.
  • Existing taxonomies focus on information-seeking subcategories rather than a general-purpose query taxonomy, making them unsuitable for this study's scope.

3 WILDSEEK

WILDSEEK is a manually annotated dataset of real human–LLM information-seeking queries, organized by intent, risk-sensitive domain, and open-endedness. It distinguishes factoid requests from analytical requests that require interpretation, synthesis, or judgment.

  • 3.1 Source Datasets: WILDSEEK contains 3,077 manually annotated information-seeking queries drawn from four in-the-wild LLM interaction datasets.The source datasets include WILDCHAT, SHAREGPT4, LMSYS-CHAT-1M, and SES.
  • 3.2 User Intent Taxonomy: The taxonomy separates information-seeking from content creation, coding, non-English queries, and no-request inputs.Info Seeking includes factoid or analytical information requests and problem-solving requiring external knowledge.
  • 3.3 Risk-sensitive Domains: WILDSEEK summarizes information-seeking queries by domain category and open-endedness type.The dataset composition is reported for 3,077 queries.
  • 3.3 Risk-sensitive Domains: Risk-sensitive domains are those where model outputs may influence decisions or actions and errors or omissions could have substantial real-world consequences.The domain definition is operationalized using prior literature on LLM safety.
  • 3.4 Open-endedness of Info-seeking Queries: Factoid queries can be answered with a single verifiable source, whereas analytical queries require reasoning beyond retrieval.Analytical queries include instructions, comparisons, predictions, and subjective judgment or advice.

4 Characterizing User Queries

The authors use WILDSEEK-trained classifiers to characterize more than 1.8 million realistic queries. Information-seeking is prevalent, risk-sensitive topics are common, and analytical requests dominate several sensitive domains.

  • Scaling Query Characterization: Classifiers trained on WILDSEEK achieve macro-F1 scores from 0.81 to 0.83 and annotate more than 1.8M deduplicated prompts.Separate ModernBERT models predict user intent, risk-sensitive domain, and open-endedness.
  • Prevalence of Information-seeking Queries: Information-seeking is the largest query category in every dataset except WILDCHAT, ranging from 42% in LMSYS-CHAT-1M to 74% in SES.Content creation dominates WILDCHAT at 56%.
  • Prevalence of High-risk Sensitive Queries: Among risk-sensitive information-seeking queries, Moral Values and Religion is most frequent at 10.1%, followed by Health at 9.1% and Economic and Financial at 7.4%.Politics and Judicial and Legal are comparatively rare, at around 2.3%.
  • Prevalence of High-risk Sensitive Queries: Risk-sensitive queries span diverse themes, including nutrition and medicines, cybersecurity and firearms, relationships and identity, legal documents, and political events.The topic clusters demonstrate broad variation in information needs across the six domains.
  • Factoid vs. Analytical Info-seeking Queries: Analytical queries are especially prevalent in Moral Values and Religion, Economic and Financial, and Security and Personal Safety domains.Judicial and Legal, Health, and Politics contain higher shares of factoid queries.

5 Framework for Evaluating LLMs

The evaluation framework assesses LLM information delivery across reliability, fairness, and safety, including factuality, source diversity, sycophancy, overreliance, and vulnerable-population handling. It compares three closed models with and without web search.

  • Framework Design: The framework is designed to apply across high-risk sensitive domains while evaluating whether responses are reliable, factual, and safe.These criteria are positioned as fundamental to information delivery.
  • Factuality: Factuality is evaluated with Loki through claim decomposition, check-worthiness assessment, search-query generation, evidence retrieval, and claim verification.Evidence retrieval uses the Google Search API through Serper API, while the remaining steps are LLM-based.
  • Diversity: Source diversity is decomposed into richness, the number of unique domains, and evenness, measured with Pielou’s J = H/ log |D|.Higher evenness indicates less concentration in a narrow set of web domains.
  • Sycophancy and Overreliance: Sycophancy measures agreement-seeking patterns, while overreliance assesses whether guidance includes appropriate disclaimers and professes limits.Sycophancy includes hollow flattery, premise acceptance, preference mirroring, caveat omission, and face-preserving validation.
  • Vulnerable Population: Vulnerable-population safety is evaluated for users including children, elderly people, people with mental illness, and those in financial difficulty.The setup assumes the model knows only cues contained in the query.
  • Evaluated Models: The study evaluates GPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Flash-Lite Preview with and without API-enabled web search.The search comparison is motivated partly by the expectation that retrieval-augmented generation may improve factuality.

6 LLM Evaluation Results

Search access does not consistently improve reliability, fairness, or safety across the evaluated models. Analytical queries produce higher failure rates than factoid queries, especially for overreliance and anthropomorphization.

  • Reliability: GEMINI-3.1 is the only model with a statistically significant factuality improvement from search, increasing by Δ=+0.049.GPT-5.4 increases by Δ=+0.006, while CLAUDE-4.6 decreases by Δ=−0.026.
  • Reliability and fairness: GPT-5.4 retrieves the most highly credible sources, whereas GEMINI-3.1 retrieves the fewest; CLAUDE-4.6 retrieves the most unique domains.GPT-5.4 nevertheless shows the least diverse retrieval behavior overall because it relies on a more concentrated source set.
  • Fairness and safety: Search does not reduce US-centric framing overall, with aggregate US Bias changing by Δ=−4.00.GEMINI-3.1 is the only model with a significant reduction in US Bias when search is enabled.
  • Fairness and safety: Search generally reduces safety failures, but improvements are inconsistent across models and are driven mainly by GEMINI-3.1.Significant reductions also occur for GPT-5.4 in overreliance and CLAUDE-4.6 in anthropomorphization.
  • Query-type effects: Analytical queries have a 4.03-percentage-point higher overall failure rate than factoid queries, with overreliance showing the largest gap at Δ=+13.96.Analytical queries significantly exceed factoid queries in 4 of 6 criteria; anthropomorphization also increases by Δ=+4.30.
  • Failure patterns: 14.31% is the highest overall failure rate for overreliance, followed by vulnerable population, US bias, and sycophancy at around 9% each.Failure rates across the four leading criteria range from 10.52% to 32.14%.

7 Discussion

Information-seeking is common, frequently high-risk, and often analytical, while the most problematic model behaviors extend beyond factual errors. Search does not consistently improve reliability, safety, or fairness, and its apparent benefits for Gemini remain uncertain.

  • Proportion of high-risk sensitive and non-factoid queries: 40% of user turns are information-seeking queries, including 37% involving high-risk sensitive domains and 60% with analytical properties.The leading high-risk topics are moral values and religion, health, and economic and financial information.
  • Risks introduced by LLMs in information-seeking: Overreliance, US bias, sycophancy, and vulnerable-population handling are the most problematic behaviors, with analytical queries generally producing more unsafe and unfair responses.The analytical-query pattern holds across all models and most setups, except for US bias.
  • Web search as a mitigation strategy: Search does not consistently increase response reliability and consistently improves safety and fairness only for Gemini.This finding invalidates the hypothesis that search would reliably improve model performance.
  • Web search as a mitigation strategy: The claim that smaller models benefit more from retrieved documents remains speculative because model size and training regimes are opaque.The relative size difference between the evaluated models is also uncertain.

8 Conclusion

WildSEEK and its evaluation framework ground discussion of LLM information-seeking behavior in real user queries rather than synthetic benchmarks. They provide a reusable starting point while leaving normative, cultural, and personalization questions open.

  • Empirical foundation: WILDSEEK and the evaluation framework are a first empirical attempt to study LLM information-seeking using real user queries rather than synthetic benchmarks.The paper frames this work as an opening move for evaluating LLM behavior as these systems mediate information access.
  • Open problems: Future work must address normative trade-offs, adaptation across cultural and linguistic contexts, and personalization versus profiling risks.The paper identifies these as open problems for the research community.
  • Future use: WILDSEEK is proposed as a longitudinal benchmark and basis for future alignment, personalization, and diversity strategies focused on high-risk and analytical queries.The paper states that analytical queries account for the majority of queries.

Limitations

The study is constrained by incomplete coverage of real-world interactions, English-only data, a limited proprietary-model evaluation, and possible LLM-based annotation errors.

  • Dataset scope: The static dataset cannot exhaustively represent all information-seeking interactions or emerging topics after collection.Its queries span broad topics but not the entire information-seeking spectrum.
  • Language and population coverage: The analysis uses English queries only and mainly reflects WEIRD speakers.The authors suggest multilingual models and translated data as future ways to broaden coverage.
  • Model coverage: The response evaluation covers three proprietary models with search access, which may affect reproducibility.The authors cite current usage, web-search functionality, and computational feasibility as selection reasons.
  • Annotation: LLM-based annotation introduces potential annotation errors despite evaluation of the judges against human-annotated ground truth.The authors explicitly acknowledge this residual risk.

Ethical Considerations

The paper addresses ethical considerations through anonymized or donated data, explicit annotation procedures, and a framework whose values and dual-use risks are acknowledged.

  • Data governance: The datasets were anonymized, and users either consented to collection or donated their data.These practices govern the construction of WILDSEEK from interaction data.
  • Normative choices: The evaluation criteria prioritize accuracy, professional accountability, and user safety, but reflect values that may not be universal.The authors note that alternative frameworks could weight user autonomy differently.
  • Annotation procedure: The annotation process uses ordered conversation turns, allowing annotators to consider preceding context but not later user turns.This defines the available conversational context for labeling.
  • Taxonomy: The taxonomy distinguishes information-seeking, content creation, coding, and no-request inputs using explicit task and content criteria.Information-seeking requires a clear request and external information, while the other categories cover generation, coding, or absent requests.
  • Annotation validation: Two authors achieved substantial agreement on query-type annotations, with average Cohen’s κ = 0.80 across rounds.The dataset was expanded with manual annotations after this validation process.
  • Risk sensitivity: Risk-sensitive domains are defined as queries that may affect life, safety, health, finances, or personal decisions.The taxonomy includes politics, finance, security, health, legal matters, moral values and religion, and other.

B.2 Further results on the risk-sensitive query domains

The paper develops and validates taxonomies for risk-sensitive domains and open-ended query types, then trains ModernBERT classifiers to scale analysis across in-the-wild datasets.

  • Risk-sensitive domains: Risk-sensitive domains span politics, finance, personal safety, health, legal matters, moral values and religion, and other topics.The taxonomy was created because prior work did not cover the full range of safety-sensitive domains in information-seeking.
  • Query types: Factoid queries seek verifiable information, whereas analytical queries require reasoning, interpretation, synthesis, or judgment.Procedural, predictive, comparative, and advice-seeking queries are classified as analytical.
  • Manual annotation: Three annotators labeled 3,123 queries after guideline training and calibration against author-produced ground truth.Majority voting produced labels for 944 cases, while an author resolved 46 cases without a majority.
  • Query-type classification: ModernBERT-large with maximum length 256 achieved information-seeking F1-macro = 0.90 and across-labels F1-macro = 0.82.The setup used supervised fine-tuning and 5-fold cross-validation after zero-shot approaches were unsatisfactory.
  • Risk-domain classification: The best risk-sensitive-domain classifier reached average F1-macro = 0.81 across folds, with Health F1 = 0.87.The large model with maximum length 256 performed best, while performance varied across categories.
  • Open-endedness classification: The best open-endedness classifier reached average F1-macro = 0.83 across folds.Results compare ModernBERT sizes and maximum input lengths in cross-validation.

F.2 Evaluation of the LLM-as-a-judge

The paper evaluates LLM-as-a-judge reliability and defines binary rubrics for response quality, including factuality, sycophancy, overreliance, vulnerable-population safety, US-bias absence, anthropomorphism, and dual use.

  • Judge validation: QWEN3.6-27B had very low recall, particularly for the unsafe category, so GPT-5.4-MINI was selected as judge.The authors describe the weaker judge as less trustworthy for evaluation.
  • Scoring: All evaluation dimensions use binary scores: 1 when a criterion is met and 0 when it is not.The rubrics were presented verbatim to annotators.
  • Sycophancy: Sycophancy measures whether responses sacrifice accuracy or honesty to tell users what they want to hear.The rubric checks flattery, premise acceptance, preference mirroring, omitted caveats, and face-preserving validation.
  • Overreliance: Overreliance prevention requires prominent, specific disclaimers when responses substitute for professional judgment.General conceptual explanations without personalized application can receive a passing score without disclaimers.
  • Vulnerable populations: Vulnerable-population safety evaluates whether responses protect children, older adults, people with mental illness, and financially vulnerable users.Health content should avoid harmful specificity, while financial content should not encourage risky behavior without acknowledging suitability concerns.
  • US bias: US-bias absence evaluates whether responses avoid applying US institutions, laws, currencies, or norms as universal when jurisdiction is unknown.Explicitly identifying US-specific information or prompting local verification can avoid failure.
Loading 2608.30683v1…