PaperlayerView exact revision on arXiv

Source-linked AI summary

Conversational Capture: A Trajectory-Level Framework for Evaluating Generative Engine Optimization in Multi-turn Human-Agent Interaction

Junwei Yu, Jieyu Zhou, Mufeng Yang, Yepeng Ding, Hiroyuki Sato

arXiv:2609.40069v1cs.HCcs.IR

TL;DR

Single-turn GEO evaluation misses how answers shape later questions and retrieval, making conversational impact path-dependent. The paper formalizes this feedback as conversational capture and develops trajectory-level measures; theory and a model-derived illustration show feedback can dominate direct gain and single-turn rankings can diverge from trajectory rankings.

  • Problem

    Single-turn GEO evaluation treats queries as exogenous and turns as independent, although answers can shape subsequent questions and retrieval in conversation.

    Source: §1 · p. 2

  • Method

    The paper models a two-layer human-agent loop and separates trajectory feedback into machine-side and human-side capture channels.

    Source: §1 · p. 2, §4.3 · p. 4, §5.3 · p. 4

  • Results

    Feedback can exceed the direct term in the model-derived illustration, and single-turn and trajectory rankings can diverge.

    Source: §7.2 · p. 6, §11 · p. 8

  • Takeaways & Limitations

    The paper concludes that evaluation should measure trajectories, while answer engines should re-ground, diversify, and explain persistent citations.

    Source: §11 · p. 8

    The reported values are model-derived, not a substitute for empirical validation of the human-side channel with deployed systems and real users.

    Source: §10 · p. 8

Abstract

from arXiv · show

Generative Engine Optimization (GEO) shapes content to increase its likelihood of being cited by answer engines built on retrieval-augmented large language models. GEO is typically evaluated as a single-turn property: for a fixed query, an evaluator measures a source's visibility in one answer. We argue that the single answer is an inadequate unit of analysis. Human-agent information seeking forms a closed loop: the agent's answer changes the user's beliefs and therefore the next question, which in turn determines what the agent retrieves. We introduce conversational capture, a phenomenon in which a source cited early becomes substantially more likely to be cited again. Capture operates through a machine-side channel, history-conditioned retrieval, and a human-side channel, follow-up questions directed toward the captured source. We formalize the interaction as a two-layer closed-loop system and derive trajectory-level constructs: cumulative conversational visibility; a direct/feedback decomposition of trajectory gain; a nested split of the feedback term into machine-side and human-side channels; a capture coefficient; a compounding ratio; and a misranking diagnostic. Using reinforcement-process (P\'olya-urn) theory, we prove that the feedback term is zero under single-turn evaluation and that GEO's cumulative payoff grows superlinearly with conversation length while capture develops. A model-derived illustration shows that the feedback term can exceed the direct term, the compounding ratio exceeds two within ten turns, and single-turn and trajectory rankings agree only weakly (Kendall's $\tau = 0.4$). We connect the human channel to information foraging, trust calibration, and Bayesian persuasion, and discuss design implications for answer engines.

1 Introduction

The paper argues that single-turn GEO evaluation misses how an answer can shape later questions and retrieval, making GEO’s impact path-dependent. It introduces conversational capture and a trajectory-level framework, supported by theory and a model-derived illustration, while identifying empirical validation as future work.

1 Introduction

  • Single-turn evaluation treats queries as exogenous and turns as independent, although conversational answers influence the user’s next question.The paper argues that this misses GEO’s cumulative, path-dependent influence across interactions.Source: §1 · p. 2
  • Conversational capture describes how an early citation can raise a source’s later citation probability through history-conditioned retrieval and user-directed follow-up questions.The persistence can become decoupled from the user’s current need, which may have narrowed or shifted.Source: §1 · p. 2
  • The paper formalizes the interaction as a two-layer loop and defines trajectory-level measures, including cumulative visibility, channel-decomposed gain, capture, compounding, and misranking.The framework separates machine-side and human-side feedback and connects the human channel to information foraging, trust calibration, and Bayesian persuasion.Source: §1 · p. 2
  • Reinforcement-process theory establishes that feedback vanishes under single-turn evaluation and that cumulative GEO gain grows superlinearly while capture develops.The model-derived illustration shows that rankings by single-turn and trajectory gain can diverge (Kendall’s τ=0.4).Source: §1 · p. 2
  • The paper offers theory and modeling rather than a new human study, and identifies deployed retrieval systems and real users as needing empirical validation.Its computational illustration is a controlled calculation from the model’s dynamics.Source: §1 · p. 3

2 Motivation: Why Single-Turn GEO Evaluation Fails

Single-turn GEO evaluation assumes queries are independent of the engine and turns are independent. In conversation, answers shape follow-up questions and history-conditioned retrieval, allowing citation advantages to persist.

2.1 The implicit assumptions of current evaluation

  • Single-turn GEO evaluation assumes queries are exogenous and turns independent, scoring average visibility lift over independently sampled queries.The procedure measures source prominence in an answer using visibility metrics such as position-adjusted word count or an LLM-judged impression score.Source: §2.1 · p. 3

2.2 The multi-turn reality: a closed loop

  • An answer can shape the user’s next question, while also making subsequent retrieval more likely to feature the same source.These coupled effects raise the chance of another citation and feed back into the interaction.Source: §2.2 · p. 3

2.3 Conversational capture

  • Conversational capture is the elevated likelihood that an earlier-cited source appears again, potentially regardless of its relevance to a narrowed or shifted need.Dialogue-history conditioning, consistency with prior answers, and conversationally biased retrieval are identified as mechanisms [20].Source: §2.3 · p. 3

3 Positioning and Related Work

Prior GEO work measures visibility per answer, while this paper argues that conversational evaluation must account for how treatment can change later questions. It distinguishes this issue from merely scoring more turns and connects it to research on search bias and information seeking.

GEO and adversarial visibility

  • Prior GEO work optimizes source content for generative engines and measures visibility per answer, including work on manipulated product visibility in LLM answers [2] [11].This paper instead examines whether a single turn is the appropriate evaluation unit for conversational settings.Source: §3 · p. 3

Conversational search and retrieval-augmented generation

  • Conversational QA and RAG establish that retrieval can depend on dialogue context, but fixed prerecorded query replays leave later questions exogenous [1] [3] [18] [10] [13].In replayed sequences, the human-side feedback term is present but not separated from the direct term.Source: §3 · p. 3

Search bias, manipulation, and generative echo chambers

  • Prior work examines how ranking and presentation shape beliefs and how LLM-powered search can narrow information seeking.This paper makes the turn-to-turn mechanism explicit and ties it to a quantitative evaluation gap for GEO.Source: §3 · p. 3

4 A Model of Multi-turn GEO

The model represents GEO as a two-layer interaction between an agent that assigns per-turn source visibility and a user whose beliefs shape queries. It separates capture into measurable machine-side and human-side channels.

4.1 Objects

  • The model’s agent layer assigns each candidate source per-turn visibility using an existing GEO metric, while the user layer updates beliefs after reading the answer and generates the next query.The user state, query, answer, and update operators define the interaction’s two layers.Source: §4.1 · p. 3

4.2 Closed-loop dynamics

  • GEO biases the agent, and the analysis traces how that initial bias propagates through the closed-loop dynamics.The dialogue history contains prior queries and answers; Figure 1(A) depicts the loop.Source: §4.2 · p. 3

4.3 Two independent capture channels

  • Machine-side capture arises when history-conditioned retrieval makes an earlier-cited source more likely to recur even on fixed queries, whereas human-side capture shifts beliefs and follow-up queries toward that source.M1 can be measured by comparing history-conditioned and stateless engines on identical queries; M2 isolates the contribution of human question-asking behavior.Source: §4.3 · p. 4

5 Trajectory-Level Evaluation Constructs

The section defines trajectory-level measures from per-turn visibility to capture cumulative exposure, feedback beyond fixed-path lift, and how capture changes method rankings. It separates history-conditioned and user-query channels and provides diagnostics for reinforcement and compounding.

5 Trajectory-Level Evaluation Constructs

  • Trajectory-level evaluation constructs are computable from the per-turn visibilities an engine produces along a conversation.The framework uses the same per-turn visibility values as existing pipelines.Source: §5 · p. 4

5.1 Cumulative conversational visibility (CCV)

  • Cumulative conversational visibility sums a source’s visibility across turns, with γ_t weighting attention or salience.Setting γ_t = 1 gives raw cumulative visibility; increasing or decreasing weights can represent recency or primacy.Source: §5.1 · p. 4

5.2 Trajectory gain and its decomposition

  • Trajectory gain compares GEO and no-GEO cumulative visibility along the realized conversation and decomposes into direct and feedback terms.The direct term sums lift along the no-GEO counterfactual path; the feedback term captures visibility from GEO-shifted queries and history, and is zero under single-turn evaluation or a memoryless engine.Source: §5.2 · p. 4

5.3 Separating the two channels:

  • The feedback term separates into machine-side history conditioning and human-side query drift through nested counterfactuals.The machine-side component can be measured by comparing history-conditioned and stateless engines on identical queries; the human-side component is the residual and includes channel interaction.Source: §5.3 · p. 4

5.4 Capture coefficient

  • The capture coefficient κ(s) measures whether a prior citation increases the probability of the source being cited next.A positive coefficient is the operational signature of capture.Source: §5.4 · p. 5

5.5 Compounding ratio

  • The compounding ratio compares realized trajectory gain with the linear extrapolation from single-turn gain; ρ > 1 means GEO compounds across turns.A ratio of 1 corresponds to independent turns, when single-turn evaluation is exact.Source: §5.5 · p. 5

5.6 Misranking diagnostic

  • Kendall’s τ compares GEO-method rankings by single-turn gain and trajectory gain; low or negative values indicate decision-relevant ranking disagreement.The diagnostic identifies when the single-turn “best” method differs from the one that captures conversations.Source: §5.6 · p. 5

6 Theoretical Analysis: Path Dependence and Superlinear Gain

The theoretical analysis models citation capture as a reinforcement process and derives conditions for path dependence, superlinear gains, and feedback. Its conclusions depend on reinforcement conditions: confirmatory feedback is nonnegative, while corrective follow-ups can make it negative.

Path dependence / non-ergodicity

  • Under symmetric linear reinforcement, citation share converges almost surely to a random limit determined by initial composition, so early turns influence the long-run outcome.Raising the GEO source’s initial share shifts the limit distribution upward.Source: §6 · p. 5

Superlinear cumulative gain

  • With asymmetric reinforcement favoring the GEO source, trajectory gain is strictly convex while capture develops, then becomes asymptotically linear as visibility saturates.The compounding ratio exceeds 1 and rises toward a finite ceiling.Source: §6 · p. 5

The feedback term is the loop

  • Under confirmatory reinforcement, the feedback term is nonnegative and is zero only with symmetric reinforcement, a memoryless engine, and no query drift.If users instead ask corrective follow-ups, the human channel and total feedback can be negative, so single-turn evaluation overestimates GEO.Source: §6 · p. 5

Capture and channel decomposition

  • The capture coefficient is positive under asymmetric reinforcement, and GEO increases it by increasing the GEO source’s reinforcement.The channel-additive condition makes the nested feedback terms nonnegative; each channel’s term is zero under its corresponding null condition.Source: §6 · p. 5

Channel interaction and scope

  • The channel terms exhaust feedback and interact super-additively while capture develops, then sub-additively as visibility approaches its ceiling.The sign of feedback is an empirical assumption: the protocol is intended to determine which regime dominates in deployed systems.Source: §6 · p. 5

7 Computational Illustration

A controlled model illustration shows that conversational feedback can increase cumulative GEO gain and change method rankings relative to single-turn evaluation. Its qualitative results hold under stated model conditions, while their magnitude depends on the selected parameters and conversation length.

7 Computational Illustration

  • The computation demonstrates that the framework’s constructs are computable, but its values do not estimate effects in deployed engines.The authors identify empirical validation with retrieval systems and real users as future work.Source: §7 · p. 6

7.1 Setup

  • The worked example sets δ=0.12, β=0.6, and γ_t=1 for T=10; β combines machine-side and human-side reinforcement.The illustration uses β_M1=0.35 and β_M2=0.25; β=0 corresponds to a memoryless engine.Source: §7.1 · p. 6

7.2 Computed constructs and predicted behavior

  • The closed-loop visibility share rises from 0.62 at turn 1 to 0.91 at turn 10, while the direct-only counterfactual stays at 0.62.Figure 2’s shaded gap represents the feedback contribution.Source: §7.2 · p. 6, Figure 2 · p. 6
  • At T=10, trajectory gain 2.68 splits into direct gain 1.20 and feedback gain 1.48, with machine-side 0.85 and human-side 0.63 contributions.The feedback term exceeds the direct term, and the compounding ratio is 2.23.Source: §7.2 · p. 6, Table 1 · p. 7

7.3 Misranking under single-turn evaluation

  • At T=10, single-turn and trajectory rankings have Kendall’s τ=0.4: method A falls from first to third, while method B rises from second to first.The five-method comparison is shown in Table 2.Source: Table 2 · p. 7, §7.3 · p. 7

7.4 Robustness of the qualitative claims

  • Across the model’s parameter conditions, feedback is positive and compounding exceeds one when capture strength β>0; rankings can diverge when salience and capture are not perfectly aligned.In 5 × 10^4 random sets, rankings disagreed in 90% of draws; with negatively correlated β and δ, the single-turn winner was wrong in 91%.Source: §7.4 · p. 7

8 Human-Computer Interaction Foundations

The paper connects the human-side channel of conversational capture to information foraging, trust calibration, and Bayesian persuasion. These theories specify mechanisms and concepts for interpreting the user’s role without assuming manipulation.

Information foraging

  • Information foraging frames follow-up queries as moves along perceived information scent, which a GEO-shaped answer can redirect toward a source’s region.The paper presents this account as a concrete, testable interpretation of the query operator.Source: §8 · p. 7

Trust calibration

  • Trust calibration distinguishes perceived trustworthiness from actual reliability; repeated exposure may increase perceived reliability while reducing exposure to alternatives for recalibration.The paper defines model-layer miscalibration as the gap between citation share and ground-truth reliability share.Source: §8 · p. 7

Bayesian persuasion

  • Bayesian persuasion frames multi-turn GEO as a sequential signaling game among a content owner, an agent, and a user updating a posterior.This framing supplies concepts of welfare, manipulation, and commitment without assuming that users are manipulated.Source: §8 · p. 7

9 Discussion: Implications for Answer-Engine Design

The discussion recommends focusing defenses on early turns, targeting both capture channels, and evaluating GEO over trajectories rather than single answers.

Conversational capture as a manipulation surface

  • Because capture leverage is concentrated in the earliest turns, the paper recommends focusing defenses there.Proposition 6.1 supports this emphasis on early turns.Source: §9 · p. 7

Design levers

  • The model suggests per-turn re-grounding, enforced source diversity, and provenance transparency as mitigations targeting retrieval history, source share, and trust recalibration.Re-grounding reduces the weight of dialogue history; provenance helps distinguish prior citation from current relevance.Source: §9 · pp. 7–8

Evaluation should move to trajectories

  • GEO audits, benchmarks, and abuse detection should report trajectory-level quantities and ranking misalignment alongside single-turn visibility.The proposed constructs use per-turn visibility values already produced by existing pipelines.Source: §9 · p. 8

10 Limitations and Threats to Validity

The section bounds the paper’s proposed empirical study and model, identifying where simulation, behavioral validation, measurement, and source-count limits apply.

Dependence on user simulation

  • Only the M2 counterfactual requires a simulated branch; the main trajectory uses real human queries, and simulator sensitivity should be reported.The simulator’s follow-up distribution should match observed moves such as deepening, rephrasing, challenging, and pivoting.Source: §10 · p. 8

Behavioral assumptions

  • The user operators and trust-miscalibration construct are theory-grounded, but the model does not claim manipulation and its behavioral hypothesis needs validation.Source: §10 · p. 8

Other limitations

  • LLM-judge visibility scoring is imperfect, open-source RAG does not represent commercial closed engines, and real human trust or perception is unmeasured.The proposed study should report agreement with gold labels and inter-judge reliability, and replicate across several open models.Source: §10 · p. 8
  • The two-source urn simplifies a many-source landscape; competition among several optimized sources remains outside the model despite a stated K-source generalization under linear reinforcement.Source: §10 · p. 8

11 Conclusion

The conclusion argues that single-turn GEO evaluation misses feedback-driven gains and can misrank methods. It presents trajectory-level evaluation as the alternative, while calling for empirical validation of the human-side channel.

11 Conclusion

  • The model formalizes conversational capture, separates machine-side and human-side channels, and defines trajectory-level evaluation constructs.Source: §11 · p. 8
  • At T=1, the feedback term is zero; as conversation length grows, cumulative GEO gain grows superlinearly and single-turn extrapolation increasingly underestimates it.The model-derived computation shows feedback can dominate direct gain and single-turn and trajectory rankings can diverge.Source: §11 · p. 8
  • Because single-turn and trajectory rankings can diverge, single-turn evaluation can select the wrong GEO method.Source: §11 · p. 8
  • The authors recommend measuring trajectories and suggest re-grounding, diversifying, and explaining persistent citations; empirical validation of the human-side channel with real users remains next.Source: §11 · p. 8

Related papers

Loading 2609.40069v1…