PaperlayerView exact revision on arXiv

Source-linked AI summary

GEO: Generative Engine Optimization

Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande

arXiv:2311.09735v3cs.LGcs.IR

TL;DR

Generative engines give creators less control over how their content appears, while potentially reducing organic website traffic. The paper introduces GEO, a black-box framework for optimizing content and visibility metrics, and GEO-bench for evaluation; it reports visibility gains of up to 40%, with effectiveness varying by domain.

  • Problem

    Generative engines’ direct answers may reduce organic website traffic, and their black-box nature makes it difficult for creators to control how content is ingested and portrayed.

    Source: §1 · p. 1

  • Method

    GEO is a black-box framework for optimizing website content and creator-defined visibility metrics, evaluated using GEO-bench’s diverse queries and relevant sources.

    Source: §1 · p. 2

  • Results

    Up to 40% source visibility gains were reported, with GEO methods’ effectiveness varying by query domain.

    Source: §8 · p. 9

  • Takeaways & Limitations

    The findings indicate that visibility strategies can be combined and that GEO methods’ effectiveness depends on query domain.

    Source: §8 · p. 9

    GEO methods may need adaptation as engines and query distributions evolve; effects on search rankings were not evaluated.

    Source: §9 · p. 9

Abstract

from arXiv · show

The advent of large language models (LLMs) has ushered in a new paradigm of search engines that use generative models to gather and summarize information to answer user queries. This emerging technology, which we formalize under the unified framework of generative engines (GEs), can generate accurate and personalized responses, rapidly replacing traditional search engines like Google and Bing. Generative Engines typically satisfy queries by synthesizing information from multiple sources and summarizing them using LLMs. While this shift significantly improves $\textit{user}$ utility and $\textit{generative search engine}$ traffic, it poses a huge challenge for the third stakeholder -- website and content creators. Given the black-box and fast-moving nature of generative engines, content creators have little to no control over $\textit{when}$ and $\textit{how}$ their content is displayed. With generative engines here to stay, we must ensure the creator economy is not disadvantaged. To address this, we introduce Generative Engine Optimization (GEO), the first novel paradigm to aid content creators in improving their content visibility in generative engine responses through a flexible black-box optimization framework for optimizing and defining visibility metrics. We facilitate systematic evaluation by introducing GEO-bench, a large-scale benchmark of diverse user queries across multiple domains, along with relevant web sources to answer these queries. Through rigorous evaluation, we demonstrate that GEO can boost visibility by up to $40\%$ in generative engine responses. Moreover, we show the efficacy of these strategies varies across domains, underscoring the need for domain-specific optimization methods. Our work opens a new frontier in information discovery systems, with profound implications for both developers of generative engines and content creators.

1 INTRODUCTION

Generative engines retrieve sources and use generative models to produce responses, creating new visibility challenges for website and content creators. The introduction presents GEO as a creator-focused optimization framework, alongside customizable visibility metrics and GEO-bench, and reports visibility gains of up to 40%.

1 INTRODUCTION

  • Generative engines retrieve relevant documents and use large neural models to produce source-grounded responses with attributions, rather than merely listing relevant websites.The systems combine conventional search with generative models.Source: §1 · p. 1, Figure 2 · p. 3
  • Direct answers may reduce organic website traffic, while the black-box nature of generative engines makes it difficult for creators to control how their content is ingested and portrayed.The introduction identifies website and content creators as a third stakeholder alongside users and developers.Source: §1 · pp. 1–2
  • GEO is a flexible black-box framework that optimizes website content and lets creators define visibility metrics suited to generative-engine responses.The framework tailors presentation, text style, and content; its metrics can account for citation relevance and influence from objective and subjective perspectives.Source: §1 · p. 2
  • GEO-bench provides 10,000 diverse-domain queries and relevant sources to support systematic evaluation of GEO methods.Source: §1 · p. 2
  • Up to 40% visibility gains were reported across diverse queries; citations, relevant-source quotations, and statistics also significantly boosted visibility, with improvements up to 37% on Perplexity.ai.The introduction also reports that GEO’s efficacy varies across domains.Source: §1 · p. 2

2 FORMULATION & METHODOLOGY

Generative engines synthesize cited responses, so traditional search rankings do not capture how visible a source is. GEO proposes creator-focused black-box content optimization and impression metrics for this setting.

2.1 Formulation of Generative Engines

  • Generative engines take a query and personalized user information to produce a natural-language response, using retrieval and generative components that can reformulate queries, summarize sources, and compose a cited answer.The paper focuses on single-turn engines; it notes the formulation can extend to multi-turn conversational engines.Source: §2.1 · p. 3

2.2 Generative Engine Optimization

  • GEO frames website visibility as an impression to maximize, while the engine’s relevance-weighted visibility objective depends on black-box design and subjective measures.The paper says traditional SEO methods are not directly applicable because generative engines use language models rather than relying only on keyword matching.Source: §2.2 · p. 3

2.2.1 Impressions for Generative Engines.

  • Generative engines combine sources in one response, so source length, uniqueness, and presentation affect visibility and simple ranking metrics do not apply.Traditional search engines instead present ranked lists, where average ranking is used to represent visibility.Source: §2.2.1 · p. 3
  • The proposed impression metrics are intended to be creator-relevant, explainable, and comprehensible; Word Count measures the normalized share of response words in sentences citing a source.When a sentence cites multiple sources, its word count is shared equally across those citations.Source: §2.2.1 · p. 3
  • Figure 3 contrasts ranked, verbatim search results with structured generative responses whose interleaved citations make visibility multidimensional.The framework proposes impression metrics creators can use to gauge and optimize performance, including creator-defined metrics.Source: Figure 3 · p. 4
  • Position-adjusted word count discounts citations exponentially by position, giving earlier citations more weight, while Subjective Impression covers user-facing aspects such as relevance, influence, and click likelihood.Subjective Impression also considers uniqueness, perceived position and count, and diversity of cited material.Source: §2.2.1 · p. 4

2.2.2 Generative Engine Optimization methods for website.

  • GEO methods modify website content, ranging from stylistic changes to structured additions, without requiring knowledge of the engine’s exact design.The tested strategies include authoritative wording, statistics, keyword stuffing, source citations, quotations, simplification, fluency, unique words, and technical terms; some may require added content.Source: §2.2.2 · p. 4

3 EXPERIMENTAL SETUP

The experiments evaluate GEO methods in a retrieved-source generative engine and on GEO-bench, a diverse query benchmark. The reported table comparison shows the best methods improve both impression metrics over baseline.

3.1 Evaluated Generative Engine

  • The evaluated engine retrieves sources first and then uses an LLM to generate responses; Perplexity.ai is also used to assess method generalizability.The main setup uses Google retrieval and GPT-3.5-turbo; only the top five sources are fetched, and five responses are sampled per query.Source: §3.1 · p. 5

3.2 Benchmark : GEO-bench

  • GEO-bench contains 10,000 queries from nine sources, divided into 8,000 train, 1,000 validation, and 1,000 test queries, with 80% informational and 10% each transactional and navigational.It spans 25 domains, nine query types, and seven categorizations, with query tags manually verified for high recall and precision on the test split.Source: §3.2 · p. 5

3.3 GEO Methods

  • Nine GEO methods are compared with unmodified sources on the complete GEO-bench test split, averaging results across five random seeds.Evaluation uses Position-Adjusted Word Count and Subjective Impression; the latter comprises seven subjective aspects assessed using GPT-3.5 and normalized to match Position-Adjusted Word Count’s mean and variance.Source: §3.3 · p. 5, §3.4 · pp. 5–6

3.4 Evaluation Metrics

  • 41% on Position-Adjusted Word Count and 28% on Subjective Impression are the best-method improvements over baseline on GEO-bench; Keyword Stuffing performs poorly by comparison.Subjective Impression scores are normalized with respect to Position-Adjusted Word Count, giving similar baseline scores.Source: Table 1 · p. 6
  • Relative impression improvement compares each source’s score in an original response with its score after a randomly selected source is optimized.The selected source remains constant for a query across GEO methods; all impression metrics are normalized so citation impressions in a response sum to one.Source: §3.4 · p. 6

4 RESULTS

Across GEO-bench, content changes such as adding citations, quotations, statistics, and improving fluency or readability increased visibility, while keyword stuffing offered little to no improvement. Persuasive or authoritative tone also showed no significant improvement.

  • The evaluation used GEO-bench queries and measured visibility with Position-Adjusted Word Count and Subjective Impression.Position-Adjusted Word Count accounts for response word count and citation position; Subjective Impression combines subjective factors into an overall score.Source: §4 · p. 6
  • Top methods Cite Sources, Quotation Addition, and Statistics Addition improved Position-Adjusted Word Count by 30–40% and Subjective Impression by 15–30%.The evaluation compared GEO methods with a no-optimization baseline on GEO-bench.Source: §4 · p. 6
  • Fluency Optimization and Easy-to-Understand improved visibility by 15–30%, suggesting engines value information presentation as well as content.Source: §4 · p. 6
  • Persuasive or authoritative tone produced no significant improvement, indicating generative engines are somewhat robust to these changes.Source: §4 · p. 7
  • Keyword stuffing offered little to no improvement in generative-engine responses, unlike its use in traditional SEO.Source: §4 · p. 7

5 ANALYSIS

The analysis finds that GEO effectiveness depends on query domain and strategy, and that lower-ranked websites can benefit substantially when content is optimized. Combining strategies and making targeted content edits also improved visibility in the reported evaluations.

5.1 Domain-Specific Generative Engine Optimizations

  • GEO methods’ strongest categories vary: Authoritative helps debate-style and historical queries; Cite Sources helps factual questions; Statistics Addition helps Law & Government and Opinion; Quotation Addition helps People & Society, Explanation, and History.Source: §5.1 · p. 7

5.2 Optimization of Multiple Websites

  • For websites ranked fifth in SERP, Cite Sources increased visibility by 115.1%, while top-ranked websites’ average visibility decreased by 30.3%.The authors attribute lower-ranked sites’ larger gains to generative engines conditioning on website content rather than relying on factors such as backlinks and domain presence.Source: §5.2 · pp. 7–8

5.3 Combination of GEO Strategies

  • Fluency Optimization combined with Statistics Addition performed best, exceeding any single strategy by more than 5.5%; Cite Sources averaged 31.4% improvement when combined with other methods.The combination analysis used a 200-example test subset, so its numbers differ from Table 1.Source: Figure 4 · p. 7, §5.3 · p. 8

5.4 Qualitative Analysis

  • Representative examples show visibility gains from adding statement sources or relevant statistics, and from emphasizing text or using a persuasive style.The examples illustrate changes made with minimal effort.Source: §5.4 · p. 8

6 GEO IN THE WILD : EXPERIMENTS WITH DEPLOYED GENERATIVE ENGINE

On Perplexity.ai, GEO methods also improved visibility, with Quotation Addition performing best on Position-Adjusted Word Count. Keyword Stuffing performed worse than the baseline.

  • Quotation Addition improved Position-Adjusted Word Count by 22% over the baseline on Perplexity.ai.Source: Table 5 · p. 8, §6 · p. 9
  • Cite Sources and Statistics Addition showed improvements of up to 9% and 37% on the two metrics on Perplexity.ai.Source: Table 5 · p. 8, §6 · p. 9
  • Keyword Stuffing performed 10% worse than the baseline on Perplexity.ai.Source: Table 5 · p. 8, §6 · p. 9

7 RELATED WORK

Related work covers source-backed answer generation, retrieval-augmented language models, and traditional search optimization. The paper positions GEO as distinct in its generative-engine setting and its focus on optimizing content visibility.

  • Prior source-backed answer-generation work includes training GPT-3 to navigate the web [19] and fetching sources through search engines [17] [23] [24]; this work unifies those approaches and provides a benchmark.Kumar and Lakkaraju [11] instead use adversarial text sequences to increase product visibility, whereas GEO proposes non-adversarial strategies for website content.Source: §7 · p. 9
  • Retrieval-augmented language models fetch knowledge-base sources for tasks [3] [9] [18], while generative engines also provide answer attributions and handle query reformulation, source selection, and decisions about when to perform them.Source: §7 · p. 9
  • Traditional SEO includes on-page content and user-experience changes and off-page link building, whereas GEO addresses generative models and conversational, multimodal settings.The passage states that traditional SEO strategies do not directly apply to generative engines.Source: §7 · p. 9

8 CONCLUSION

The conclusion formalizes generative engines and presents GEO as a way to optimize content visibility within them. It reports visibility gains, introduces metrics and a benchmark, and identifies domain dependence and strategy combinations as further findings.

8 CONCLUSION

  • GEO proposes optimizing content for generative engines and introduces impression metrics and GEO-bench to evaluate visibility across diverse queries and domains.The paper also releases algorithms, datasets, and infrastructure to support further work on generative engines.Source: §8 · p. 9
  • GEO methods can boost source visibility by up to 40% in generative-engine responses.The conclusion also reports that citations, relevant-source quotations, and statistics can significantly boost visibility, and that methods' effectiveness depends on query domain.Source: §8 · p. 9
  • The conclusion reports promising results on a commercially deployed generative engine with millions of active users.It also identifies the potential of combining multiple GEO strategies.Source: §8 · p. 9

9 LIMITATIONS

The limitations section notes that GEO methods and benchmark queries may need updating as generative engines and query patterns change. The accompanying material describes conversational engines, the evaluated-engine prompt, benchmark tagging, and subjective impression metrics.

9 LIMITATIONS

  • Methods may need to adapt as generative engines evolve, GEO-bench queries may require updates over time, and subjective query labels may contain discrepancies or errors.The methods were tested on two engines, including one publicly available engine; search-ranking effects were not evaluated because the algorithms are black boxes.Source: §9 · p. 9

A CONVERSATIONAL GENERATIVE ENGINE

  • A conversational engine models input as query-response history H and defines the next response r_t+1 from that history.The turn index t identifies the conversation turn; a separate LLM may generate follow-up queries to encourage engagement and provide users more detailed information.Source: A CONVERSATIONAL GENERATIVE ENGINE · p. 11

B.1 Evaluated Generative Engine

  • The evaluated-engine subsection points to Listing 1 for the exact prompt used.The supplied passage does not specify the prompt's contents.Source: §B.1 · p. 11

B.2 Benchmark

  • GEO-bench draws queries from nine datasets and tags them across seven categories, with the automated tags noted as potentially noisy.The categories cover query difficulty, nature, genre, specific topics, sensitivity, user intent, and answer type.Source: §B.2 · p. 11

B.3 Evaluation Metrics

  • The evaluation uses seven subjective impression metrics, whose prompts are available in the public repository.Source: §B.3 · p. 11

B.4 GEO Methods

The GEO methods section evaluates nine optimization methods on GEO-bench, averaging results across five random seeds. Its table captions report improvements over baselines for the best methods, including on Perplexity.ai.

B.4 GEO Methods

  • The evaluation covers nine GEO methods on the complete GEO-bench test split and averages results over five random seeds.Source: §B.4 · pp. 11–12
  • On GEO-bench, the best methods improve over baseline by 41% on Position-Adjusted Word Count and 28% on Subjective Impression; on Perplexity.ai, the gains are 22% and 37%, respectively.Source: Table 6 · p. 12, Table 7 · p. 12
  • Statistics Addition and Quotation Addition show strong performance improvements across all metrics, while Keyword Stuffing often performs worse than baselines.Source: Table 6 · p. 12, Table 7 · p. 12

B.5 Prompts for GEO methods

  • GPT-3.5 turbo was used for all GEO-method experiments, with prompts available in the public repository.Source: §B.5 · p. 12

C RESULTS

The results passage reports the number of random seeds used and points to Table 6 for results with statistical deviations.

C RESULTS

  • Results are averaged over five random seeds, with statistical deviations presented in Table 6.Source: C RESULTS · p. 12

C.1 GEO in the Wild : Experiments with Deployed Generative Engine

The authors evaluate their GEO methods on Perplexity.ai, a deployed generative engine, using uploaded source texts and 200 test samples.

C.1 GEO in the Wild : Experiments with Deployed Generative Engine

  • The GEO methods are evaluated on the deployed generative engine Perplexity.ai using a 200-sample subset of the test set.The results are reported in Table 7.Source: §C.1 · p. 12
  • Because Perplexity.ai does not allow source URLs to be specified, the authors provide source text as file uploads.They ensure that answers use only the provided file sources.Source: §C.1 · p. 12
  • The deployed-engine evaluation results are presented in Table 7.Source: §C.1 · p. 12
Loading 2311.09735v3…