Source-linked AI summary
Search Still Matters: Information Retrieval in the Era of Generative AI
William R. Hersh
TL;DR
This perspective asks how generative AI fits into information retrieval, given varied information needs and academic concerns about source authority and traceability. It considers generative AI in the search process and concludes that, although LLMs may aid it, search systems remain essential.
Problem
Academic information seeking includes complex tasks, but chatbot answers may lack direct references and traceable source text.
Source: INFORMATION NEEDS INFORM USE OF LLMS · p. 3, CHALLENGES FOR LLMS IN SEARCH · p. 4Method
The perspective considers generative AI in the information-retrieval process through the author’s academic experience and varied information needs.
Source: INTRODUCTION · p. 2Results
The author concludes that LLMs may aid information retrieval, while search systems remain essential for critical information needs and academic use.
Source: FUTURE ROLE OF LLMS IN SEARCH · p. 5Takeaways & Limitations
For scholarly work, the author checks who wrote an item, where it was published, and whether its text is useful for the task.
Source: FUTURE ROLE OF LLMS IN SEARCH · p. 5Cited concerns about LLMs include opacity and hallucinations, diverted website traffic, and loss of learning or serendipity.
Source: CHALLENGES FOR LLMS IN SEARCH · p. 4
Abstract
from arXiv · showhide
Objective: Information retrieval (IR, also known as search) systems are ubiquitous in modern times. How does the emergence of generative artificial intelligence (AI), based on large language models (LLMs), fit into the IR process? Process: This perspective explores the use of generative AI in the context of the motivations, considerations, and outcomes of the IR process with a focus on the academic use of such systems. Conclusions: There are many information needs, from simple to complex, that motivate use of IR. Users of such systems, particularly academics, have concerns for authoritativeness, timeliness, and contextualization of search. While LLMs may provide functionality that aids the IR process, the continued need for search systems, and research into their improvement, remains essential.
INTRODUCTION
Search systems make archival knowledge widely accessible, but generative-AI chatbots and features embedded in major search engines rapidly changed the search experience. The author considers this shift through academic work and the varied tasks that lead people to seek information.
INTRODUCTION
- Generative-AI chatbots and features in Bing and Google transformed Web searching after their emergence in late 2022.Search systems had been relatively mature applications before this shift.Source: INTRODUCTION · p. 2
- The author examines search from an academic perspective shaped by teaching, writing, and evidence-based medicine.These activities involve finding references and locating evidence to support decisions and syntheses.Source: INTRODUCTION · p. 2
- Information needs range from answering direct questions and making decisions to integrative tasks such as writing and teaching.Searchers also contend with misinformation and distinguishing true content from advertising or misinformation.Source: INTRODUCTION · p. 2
INFORMATION NEEDS INFORM USE OF LLMS
Academic information seeking includes varied tasks and depends on assessing the authority and timeliness of retrieved information. Chatbots can make those judgments difficult when their answers lack traceable references or source text.
INFORMATION NEEDS INFORM USE OF LLMS
- Academics often need to locate a known paper or preprint using only its title or incomplete metadata, alongside broader information tasks.The paper also describes needs such as decision support, background information, and keeping current in a subject area.Source: INFORMATION NEEDS INFORM USE OF LLMS · p. 3
- Chatbots often provide few references, may not directly cite claims, and do not reveal the source text behind an answer.Source: INFORMATION NEEDS INFORM USE OF LLMS · p. 3
- Academics value authoritative information and often check authors, publication sources, and original research supporting claims.This includes tracing claims about research methods or results to the original research.Source: INFORMATION NEEDS INFORM USE OF LLMS · p. 3
- Timeliness matters especially in rapidly evolving fields because intermittently updated LLMs may not reflect the latest information.The passage gives generative AI in biomedical and health applications as an example of a rapidly evolving field.Source: INFORMATION NEEDS INFORM USE OF LLMS · p. 3
CHALLENGES FOR LLMS IN SEARCH
The author argues that information needs extending beyond simple questions call for more than general LLM commentary or short reference lists. Concerns include source quality, search-task diversity, model opacity, and energy use.
CHALLENGES FOR LLMS IN SEARCH
- Web information quality remains difficult to establish: link-based ranking offered an imperfect proxy, while disinformation and manipulation persist [4, 5, 6,7].The author notes that efforts to automate detection of health-information quality never fully solved the problem.Source: CHALLENGES FOR LLMS IN SEARCH · p. 4
- For decisions or scholarly synthesis, the author needs more than an LLM’s general commentary or short list of references and websites.The need is especially salient for personal-health decisions and teaching or research in science.Source: CHALLENGES FOR LLMS IN SEARCH · p. 4
- Search systems must account for varied information needs, search strategies, searchers, and bias in results [8].Shah also identifies opacity, hallucinations, diverted website traffic, and loss of learning or serendipity as concerns for LLM systems [9].Source: CHALLENGES FOR LLMS IN SEARCH · p. 4
- One study estimated that Google search with generative-AI capabilities used ten times more energy than plain Google search [10].The author raises increased energy consumption as a concern about chatbots replacing search.Source: CHALLENGES FOR LLMS IN SEARCH · p. 4
FUTURE ROLE OF LLMS IN SEARCH
The author sees generative AI as a possible aid to search, but notes that evidence about its role across information needs remains limited. For academic work, the author continues to prefer search and to assess original sources.
FUTURE ROLE OF LLMS IN SEARCH
- There is not yet a comprehensive picture of how chatbots compare with search across different information needs.Reported examples include incomplete or confabulated references in several subject areas [11, 12, 13].Source: FUTURE ROLE OF LLMS IN SEARCH · pp. 4–5
- Early studies suggest possible search augmentation, but evidence remains limited and some approaches have trade-offs.Boolean query generation improved precision at a cost to recall in systematic reviews [14]; one consumer-health study found lower correctness when prompts included supporting documents [15]. Retrieval-augmented generation and other methods are also proposed [16, 17, 18].Source: FUTURE ROLE OF LLMS IN SEARCH · p. 5
- For critical information needs, the author still turns first to search and checks authorship, publication venue, and the text’s usefulness.The author considers chatbots less valuable than original sources for lectures, papers, and other intellectual syntheses, while allowing this may change.Source: FUTURE ROLE OF LLMS IN SEARCH · p. 5