ChatShop: Interactive Information Seeking with Language Agents

1 Introduction

• WebShop can be solved as single-turn retrieval because full target and candidate information plus homogeneous interfaces permit direct relevance scoring.A retrieval system can identify the target without strategic exploration.Source: §1 · pp. 1–2
• Realistic shopping involves partial target knowledge, so conversational agents can help shoppers clarify preferences while narrowing the product space.The shopper acts as a dynamic information source, making strategic and efficient questions important.Source: §1 · p. 2
• ChatShop starts with only coarse product information, requiring agents to gather missing attributes through interaction with shoppers and the product environment.The resulting task ambiguity is amplified by the vast product space and tool usage.Source: Figure 1 · p. 1, §1 · p. 2
• Experiments benchmark agents with human and LLM shoppers and find that open-source models lag commercial models, while stronger base models do not always perform better.The results verify challenges from information needs and constrained communication channels.Source: §1 · p. 2
• LLM-simulated shoppers show similar agent failure patterns to human shoppers, supporting their use as scalable evaluation partners.The paper reports that simulated shoppers effectively recover the gap between agents observed in human studies.Source: §1 · p. 2

2.1 Information Seeking Tasks

• Unlike retrieval-oriented information-seeking tasks with artificial constraints, ChatShop grounds interaction constraints in a realistic human shopping scenario.Many prior tasks can be formulated as single-turn retrieval over external knowledge sources.Source: §2.1 · p. 2
• Compared with reference games, ChatShop uses open-ended questions and non-binary feedback rather than mostly yes-no questions over narrow attribute sets.This design better reflects interaction over a broad product space.Source: §2.1 · p. 2

2.2 Human-AI Collaboration

• Prior human-AI collaboration work uses human or simulated feedback, preference elicitation, or grounded dialogue, but ChatShop targets external product-space exploration.Its search space is broader and less synthetic than DialOp’s, while its grounding differs from approaches relying on knowledge in model weights.Source: §2.2 · pp. 2–3

3.1 Background

• WebShop evaluates product selection using title similarity and attribute coverage, with success defined as reward equal to 1.The task contains more than a million scraped Amazon candidates and supports search, product-detail inspection, and confirmation actions.Source: §3.1 · p. 3
• WebShop organizes products through layered pages, but its homogeneous interface can be automated by recording product information and reformulating the task as retrieval.The interface structure does not require substantial strategic planning.Source: §3.1 · p. 3

3.2 The Retrieval Solution

• 86.8% of BM25 searches using the full instruction return 50 products containing a full-reward product, reducing the need for strategic exploration.The result indicates that the instruction itself often provides enough information for retrieval.Source: §3.2 · p. 3
• The ranking model uses cross-attention between the goal instruction and concatenated product information, trained with a pairwise margin loss and auxiliary cross-entropy when the ground-truth product is retrieved.The model ranks products from the 50-item retrieval list.Source: §3.2 · p. 3
• 78.3% success rate and 87.2 average reward are achieved by the retrieval approach, exceeding human annotators’ reported 59.6% success rate and 82.1 average reward.These results motivate redesigning shopping evaluation around meaningful interaction.Source: §3.2 · pp. 3–4

4 ChatShop

• ChatShop hides target attributes and options, replacing detailed instructions with basic product types to create ambiguity and require information accumulation.The simplified instructions are six times shorter and contain fewer unique tokens than the originals.Source: §4 · p. 4
• The agent searches products, asks clarification questions, and selects a product, while the shopper answers using access to the target information.The agent is responsible for exploring the product space and summarizing features relevant to the shopper’s decision.Source: §4 · pp. 4–5
• The agent must identify discriminative product features and learn which questions elicit informative shopper responses.Some questions reveal critical information, whereas vague questions may be bothersome or unhelpful.Source: §4 · p. 4
• The shopper can be simulated with a language-model chatbot that provides a consistent interaction partner without requiring perfect human imitation.This enables scalable benchmarking of different agents in the agent role.Source: §4 · p. 4
• ChatShop supports open-ended text interaction and instance-based comparison, while limiting shopper responses to prevent direct revelation of the target product.The shopper is instructed to reserve information until prompted and keep responses short.Source: §4 · p. 5
• Shopper interaction is bounded because redundant questions can harm the shopping experience, while final product selection uses WebShop’s reward function.Search through BM25 is not penalized, but the number of shopper-interaction rounds is limited.Source: §4 · p. 5

5.2 Benchmarking Agents in ChatShop

• Over 30% average-reward drops occur when full product information is withheld, confirming that ChatShop creates information scarcity requiring active environmental information seeking.Agents perform well with full information but significantly worse with only basic instructions.Source: §5.2 · p. 6, Table 2 · p. 6
• ReAct significantly improves interactive performance, especially when auto q agents otherwise remain confident and reluctant to ask questions.Advanced prompting helps, but the best agent remains substantially below the non-interactive full-information baseline.Source: §5.2 · p. 6, Table 3 · p. 6
• GPT-3.5 surprisingly outperforms GPT-4 in the optimal scenario, showing that stronger base-model performance does not always yield better information seeking.Agents still struggle to use communication channels effectively, and interaction can produce performance similar to or below no interaction.Source: §5.2 · p. 6

5.3 LLM versus Human Shopper

• LLM-simulated shoppers produce agent performance consistent with human shoppers, while providing slightly more information without substantially improving precision or hit rate.The human study collected 100 shopping sessions, and the comparative evaluation found similar response quality overall.Source: §5.3 · pp. 7–8, Table 5 · p. 7, Table 6 · p. 7

5.4 Error Analysis

• Misinterpretation, insufficient information gathering, and reversion are the most common errors, while simulated and human environments reveal similar failure distributions.These patterns indicate weaknesses in strategic information seeking and long-context modeling; misleading-user errors are rare in both environments.Source: §5.4 · p. 8, Figure 2 · p. 8

• ChatShop offers an information-seeking-centered evaluation and baseline agents intended to support research on agents that explore and learn effectively.Source: §6 · p. 8

7 Limitations

• ChatShop assumes that the shopper knows the target product, unlike real-world searches where shopper knowledge may evolve during exploration.Relaxing this assumption would require real-world data about the shopper’s changing knowledge.Source: §7 · p. 9
• Fixed question budgets and end-task rewards do not measure interaction quality in successful sessions, motivating dynamic budgets or penalties for uninformative questions.Source: §7 · p. 9
• The task could be enriched by integrating multimodal information such as images and reviews.Source: §7 · p. 9

A Reinforcement Learning in ChatShop

• ChatShopBin simplifies interaction to product-list selection, attribute questions, and binary shopper feedback for studying reinforcement learning.Source: A Reinforcement Learning in ChatShop · p. 11, §A.1 · p. 11

A.1 ChatShopBin

• ChatShopBin models shopper behavior with a length-based rule: attributes outside a recognizable length range receive negative responses.Source: §A.1 · p. 11

A.2 RL Setup

• Q-learning uses BERT state representations and token-level action scores, while recommendations are represented by special tokens and sessions receive terminal rewards.Questions cost 1, successful sessions receive 5, and unsuccessful sessions receive -3.Source: §A.2 · p. 11
• The setup also evaluates REINFORCE with a value baseline based on recent episode returns.Source: §A.2 · p. 11
• Both Q-learning and policy-gradient agents approach 100% success and average reward near 2.0, but training is unstable and sample-intensive.Source: §A.2 · p. 11

A.3 Potential-based Reward Shaping

• Prioritized replay can amplify unlearnable shortcuts because random guessing succeeds 1/5 of the time and receives the highest reward.Source: §A.3 · p. 11
• ChatShopBin lacks intermediate rewards, so total reward is disconnected from progress such as securing useful information.Source: §A.3 · pp. 11–12
• A listener model estimates whether products are ruled out from conversation history, enabling potential-based reward shaping when shopper behavior is not directly observable.The listener is trained from successful trajectories and updated alongside the agent.Source: §A.3 · p. 12

• Applying reinforcement learning to ChatShop is limited by sample efficiency and the need to model complex, dynamic, uncertain shopper behavior.More sophisticated and generalizable listener-aware RL agents are left for future work.Source: §A.4 · p. 12

B.1 Data Preparation

• ChatShop gives agents coarse product goals and requires them to gather further information through shopper questions before selecting a product.Agents can search the product space with BM25 and receive product titles, attributes, and options as search results.Source: §B.2 · p. 12, §B.1 · p. 12

B.2 Prompt Engineering for Agent

• The agent prompt supports repeated BM25 searches and asks agents to continue questioning until the shopper’s criteria identify a single product.OpenAI models receive 20 products per search, whereas CODELLAMA receives 5 because of response-time and long-context limitations.Source: §B.2 · p. 12
• The implementation uses zero-shot prompts and response parsing because multi-turn context and constrained model windows make few-shot prompting impractical.The authors note that a 4,096-token context length is insufficient for many language models on this task.Source: §B.2 · p. 12

B.3 Prompt Engineering for Shopper

• Shopper responses are limited to less than 5 words and cut off beyond 10 tokens, encouraging agents to ask more strategically and optimize information density per interaction turn.Source: §B.3 · p. 13
• The shopper is given the target product’s title, attributes, and options but is instructed not to reveal it directly, instead answering consistently with the target product.The response-length constraint is intended to prevent direct disclosure and make each question more informative.Source: §B.3 · p. 13

B.4 Sample Conversation

• The sample conversations show agents asking about product attributes and shoppers replying with concise preferences such as color, mounting style, size, and accessories.The examples cover hair chalk and storage cabinets, with responses progressively narrowing the desired product.Source: §B.4 · pp. 13–14

B.5 Human Evaluation

• Human evaluation recruits eight university students who act as shoppers for half- to one-hour sessions and receive average compensation of $12 per hour.These participants provide the human-shopping comparison used in the study.Source: §B.5 · p. 14

B.6 Error Types Classification

• GPT-4 classifies failed trajectories using the conversation, selected product, goal product, and separate title and attribute-or-option rewards as evidence.Error types are treated as a multi-label classification problem.Source: §B.6 · p. 14

C.1 Shopper Prompt

• The shopper prompt prevents explicit product-name disclosure while allowing indirect descriptions tied to the intended product.The prompt provides the product name and important attributes to the simulated shopper internally.Source: §C.1 · p. 14

C.2 Agent Prompt

• The sample agent prompt uses explicit action formats for search, candidate listing, questioning, and selection to structure interaction with the shopper.Selection is reserved for cases where the criteria clearly match a single product.Source: §C.2 · p. 14
• The agent can search with BM25, present candidate products, ask clarification questions, and select an item once one product matches the user’s criteria.Search can be repeated as requirements evolve, while price cannot be searched.Source: §C.2 · p. 14

C.3 Subject Extraction Prompt

• Subject extraction removes product attributes to produce a cleaned target query, while attribute removal deletes specified attributes while preserving fluency.The prompts require each component to return only the cleaned query.Source: §C.3 · p. 14, §C.4 · p. 15