Source-linked AI summary
Learning to Attend, Copy, and Generate for Session-Based Query Suggestion
Mostafa Dehghani, Sascha Rothe, Enrique Alfonseca, Pascal Fleury
TL;DR
In session-based search, users reformulate queries, but noisy session contexts can make it difficult to identify which parts should guide a suggestion. ACG combines query-aware attention with copying and generation, and outperforms baselines in both candidate scoring and query generation.
Problem
Session-based query suggestion must identify useful context amid noisy words from topic switches, navigational queries, and nondiscriminating terms.
Source: §4.3 · p. 8Method
ACG augments seq2seq with query-aware attention, a copy mechanism for session terms, a generator, and metrics for evaluating query generation.
Source: §1 · p. 2Results
Takeaways & Limitations
ACG can serve both as a model for scoring candidate suggestions and for generating query suggestions.
Source: §1 · p. 2When spelling correction is removed during preprocessing, ACG can repeat spelling errors while copying terms in spelling-correction reformulations.
Source: §4.3 · p. 9
Abstract
from arXiv · showhide
Users try to articulate their complex information needs during search sessions by reformulating their queries. To make this process more effective, search engines provide related queries to help users in specifying the information need in their search process. In this paper, we propose a customized sequence-to-sequence model for session-based query suggestion. In our model, we employ a query-aware attention mechanism to capture the structure of the session context. is enables us to control the scope of the session from which we infer the suggested next query, which helps not only handle the noisy data but also automatically detect session boundaries. Furthermore, we observe that, based on the user query reformulation behavior, within a single session a large portion of query terms is retained from the previously submitted queries and consists of mostly infrequent or unseen terms that are usually not included in the vocabulary. We therefore empower the decoder of our model to access the source words from the session context during decoding by incorporating a copy mechanism. Moreover, we propose evaluation metrics to assess the quality of the generative models for query suggestion. We conduct an extensive set of experiments and analysis. e results suggest that our model outperforms the baselines both in terms of the generating queries and scoring candidate queries for the task of query suggestion.
1 INTRODUCTION
Session-based query suggestion must account for the ordered context of a search session and users’ frequent retention of rare or unseen terms. The paper introduces ACG, which combines query-aware attention, copying, and generation, and reports better query generation and candidate scoring than baseline models.
1 INTRODUCTION
- Query suggestions can help users disambiguate queries and express needs when they lack terminology, and may accelerate search satisfaction by deepening or redirecting a search.The cited prior work associates query suggestion with both continuing a search direction and moving to another aspect of a task.Source: §1 · p. 1
- Search-log co-occurrence methods face sparsity and limited coverage for rare or unseen queries, while session context and query order provide important reformulation information.The paper describes highly diverse sessions as especially challenging for co-occurrence-based models.Source: §1 · p. 1
- Query reformulation retains an average of 62% of a query’s terms, and more than 67% of retained terms fall in the bottom 10% by frequency.Retained terms are described as core indicators of information need.Source: §1 · p. 1
- Generic word-based seq2seq models ignore query-level structure and struggle with OOV and rare terms, limiting their ability to model term retention.The paper notes that term retention is the most common query-reformulation pattern.Source: §1 · p. 2
- ACG adds query-aware attention to focus on promising session context and a copy mechanism to reuse source terms, while retaining a generator for new words.The model is trained with a multi-objective learning process.Source: §1 · p. 2
- ACG generates better queries and scores good candidates more effectively than baseline models for session-based query suggestion.The model is evaluated on AOL query-log data in both discriminative and generative settings.Source: §1 · p. 2
2 OUR PROPOSED MODEL
The model extends an attention-based sequence-to-sequence architecture with query-aware attention and copy/generate decoding for session-based query suggestion. Its training separates the generator, copier, and switch-gate updates.
2.1 Seq2seq with Attention
- The base seq2seq model uses an encoder to represent the source sequence and a decoder with attention to predict target tokens from a dynamically updated context.The source encoder is bidirectional, and attention weights determine each source state’s contribution to the target token at each decoding step.Source: §2.1 · pp. 2–3
2.2 Overview of Proposed Changes
- Hierarchical attention creates query-aware attention to control which session queries inform generation and to detect session boundaries; 62% of session-query terms are retained from earlier queries and are mostly rare.The decoder also accesses source words through a copy mechanism, while the generator can produce vocabulary words.Source: §2.2 · p. 3
2.3 Qery-Aware Attention
- A bidirectional query-level encoder represents each query, and its attention weights are combined with word-level attention and renormalized to form query-aware weights.Query representations are built from the word-level states at query-ending markers.Source: §2.3 · pp. 3–4
2.4 Copy and Generate for Decoding
- Decoding combines vocabulary generation with a copy distribution over source words, using a switch gate to choose which component supplies each prediction.The copy mechanism uses encoder states; an unknown-token option supports words that must be generated but are absent from the input.Source: §2.4 · p. 4
2.5 Multi-Objective Training
- Separate-stage parameter updates empirically perform better than combining the generator, copier, and switch-gate losses into one loss.Training defines losses for the generator and copier and rules for switch-gate targets.Source: §2.5 · pp. 4–5
3 EVALUATION
The evaluation assesses the model both as a candidate-query scorer and as a query generator. It introduces word-, embedding-, and retrieval-based measures for generated-query quality.
3 EVALUATION
- Evaluation uses two paradigms: scoring candidate queries as a discriminative model and measuring generated-query quality with proposed generative metrics.The discriminative setup adds the model’s score as a feature to a learning-to-rank framework and measures ranking quality with MRR.Source: §3 · p. 5, §3.1 · p. 5
3.1 Evaluation based on Discrimination
- The candidate-query scoring comparison adds the model’s score to the BaseRanker features while using candidates generated by a co-occurrence model.The setup replicates Sordoni et al.’s experimental design for comparison.Source: §3.1 · p. 5
3.2 Evaluation based on Generation
- Generation metrics include PER for order-independent word overlap and embedding-based cosine similarity to assess semantic matching.The embedding metric uses query-level vector-extrema embeddings built from pretrained word vectors.Source: §3.2 · p. 5
- Retrieval-based metrics compare results retrieved for generated queries with reference rankings from target queries, PRF-expanded targets, or later session queries.The latter two proposed metrics aim to evaluate generated suggestions against user information needs when user judgments are unavailable.Source: §3.2 · p. 5
4 EXPERIMENTS AND RESULTS
The experiments compare ACG with baseline models on candidate-query scoring and query generation. ACG outperforms the baselines on reported generation metrics and improves candidate scoring; analyses examine session length, noisy data, and multiple suggestions.
4.1 Dataset and Experimental Setups
- The AOL dataset contains about 9 million queries in 3 million sessions, split into 70% for model training, 20% for ranking training, and 10% for testing.Sessions use a 30-minute idle-time boundary; the logs cover three months and 657,426 unique users.Source: §4.1 · p. 6
- The discrimination setup ranks top-20 co-occurrence candidates with LambdaMART, while the experiments also compare full ACG with query-aware-attention-only and copy-only variants.BaseRanker uses the target query as relevant and other candidates as non-relevant; the component variants test query-aware attention and the copy mechanism separately.Source: §4.1 · p. 6
4.2 Main Results
- Adding query-aware attention to the copy-equipped model significantly improves performance, although attention-only hierarchical variants do not significantly outperform the simpler attention model.For retrieval evaluation, BaseRanker with ACG performs best on simret, while generative ACG performs best on sim++ret.Source: §4.2 · p. 7
- ACG has the lowest PER error among the compared generative models and outperforms all baselines on simemb; adding its score also yields the highest improvement in candidate scoring.PER is based on exact word overlap; the authors report that, on average, 38% of target-query terms are retained from earlier queries in a session.Source: §4.2 · pp. 6–7, Table 2 · p. 7
4.3 Discussions and Analysis
- ACG performs best and remains stable across session lengths, whereas HRED appears to fail on long sessions; generated-query quality generally declines at later ranks.The second-ranked query has a slightly higher sim++ret score than the first, but the increase is not statistically significant; the authors suggest later suggestions may relate to the user's ultimate information need.Source: §4.3 · pp. 7–8
- On noisy data, ACG significantly outperforms HRED and seq2seq in simemb across tested conditions and controls performance loss when unrelated sessions are inserted.ACG's main reported failure occurs when spelling correction is removed: copying can repeat an earlier spelling error.Source: §4.3 · pp. 8–9
5 RELATED WORK
Prior query-suggestion work used query-log co-occurrences, clicks, session structure, and query generation, while the paper positions query-aware attention and copying as additions to hierarchical session modeling.
5 RELATED WORK
- Query-log co-occurrence and click-based methods can suffer from sparsity, while several alternative approaches target long-tail queries or generate suggestions from logs and external resources.The related work also describes query-flow graphs, similarity methods, suffix trees, and learning-to-rank pipelines.Source: §5 · p. 9
- Compared with Sordoni et al. [42], the paper adds query-aware attention for session structure and a copy mechanism for query reformulation.The authors report that these additions lead to significant performance improvement.Source: §5 · p. 9
- Prior hierarchical models encode structure at word and sentence or utterance levels; this paper instead learns query- and word-level attention separately and integrates them during decoding.The paper relates its query-aware attention to Yang et al..Source: §5 · p. 9
- The paper relates its copy mechanism to pointer networks and prior copying-and-generation work, while learning to copy, generate, and choose between them as separate tasks.The approach is described as close to Gu et al. [16] and See et al..Source: §5 · p. 9
6 CONCLUSION AND FUTURE WORKS
The paper concludes that its query-aware-attention and copy-augmented seq2seq model improves query suggestion in both discrimination and generation, and outlines extensions using clicks, document content, and query prefixes.
6 CONCLUSION AND FUTURE WORKS
- The proposed model augments seq2seq with query-aware attention and copying, proposes generative evaluation metrics, and outperforms baselines in discrimination and generation.The conclusion describes copying as a way to model term retention in query reformulation.Source: §6 · p. 9
- Future evaluation could use retrieved-document content to estimate whether generated queries provide relevant content for users’ information needs.The authors also propose integrating clicked documents as additional generation signals.Source: §6 · p. 9
- The authors propose testing query auto-completion by using a character-based seq2seq model that conditions on session queries and the new query’s prefix.The decoder would generate the most probable sequence of following characters.Source: §6 · p. 10