PaperlayerView exact revision on arXiv

Source-linked AI summary

When More Data Is Not Enough: The Context-Sufficiency Frontier in Generative AI Personalization

Merieme Askour, Ayoub Merimi

arXiv:2610.00654v1cs.AIcs.LG

TL;DR

Customer evidence can indicate what a person may value, but it does not establish what is appropriate in a specific interaction; generative AI also makes it easy to add context that may not help. The paper develops a theory and tests it in a full-factorial recommender experiment, finding that relevant context improved appropriateness while irrelevant context reduced it and destabilized retrieval.

  • Problem

    Customer evidence alone cannot determine an appropriate response to a current request, while easy-to-supply context may be irrelevant or conflicting.

    Source: §1 · pp. 2–3

  • Method

    The paper develops a theory of context sufficiency and tests five signals across all 32 combinations in a full-factorial offline experiment with a generative recommender.

    Source: §1 · p. 3

  • Results

    Relevant context improved appropriateness, while irrelevant context reduced it and destabilized retrieval in the retailer experiment.

    Source: §1 · p. 3, §9 · p. 30

  • Takeaways & Limitations

    Personalization should supply the context an interaction needs and enforce applicable constraints throughout the service process.

    Source: §9 · p. 30

    The evidence is bounded by two constructed customer profiles making one request, one full-factorial open-weights model, and five signals representing three of five context types.

    Source: §8 · pp. 28–29

Abstract

from arXiv · show

Personalization has long relied on customer data to infer what an individual is likely to value. We call this customer evidence: the customer's historical behavior and preferences. Generative AI extends personalization by allowing providers to supply changing situational information at the moment a response is produced, without encoding every condition in advance. We define this provider-side context as information about what is possible, permitted, or advisable now. This flexibility creates a new problem: once context becomes easy to supply, more is not necessarily better. We develop a theory of context sufficiency in which the relevance of context to the customer's current intent matters more than its volume. The theory identifies four states, insufficiency, sufficiency, saturation, and interference, and introduces the Context-Sufficiency Frontier to locate the minimal relevant set. In a full-factorial experiment with a generative recommender at a large home-furnishing retailer, relevant context improved appropriateness, while irrelevant context reduced it and destabilized retrieval. The framework shifts personalization from supplying more context toward identifying what the current interaction actually requires and enforcing constraints throughout the service process.

1 Introduction

Customer evidence alone cannot establish what is appropriate in a particular interaction, and provider-side context matters only when relevant to the current request. The paper develops a context-sufficiency framework and tests it in a generative recommender experiment, finding that relevant context helps while irrelevant context can harm recommendations.

1 Introduction

  • Provider-side context is situational information that changes what response is appropriate to the customer’s current request.A fact can be decisive in one interaction and irrelevant in another.Source: §1 · p. 2
  • The paper distinguishes five provider-side context types—environment, goal, operational, world, and guardrail—whose value depends on the current request.A signal matters only when it bears on that request.Source: §1 · p. 3
  • Generative AI allows providers to supply changing context when producing a response, but additional signals may be relevant, redundant, irrelevant, or conflicting.The paper frames selecting context for the interaction as part of managing AI-delivered service.Source: §1 · p. 3
  • The Context-Sufficiency Frontier identifies the highest appropriateness attainable for a given number of signals and the smallest set needed to reach it.The theory distinguishes insufficiency, sufficiency, saturation, and interference as context accumulates.Source: §1 · p. 3
  • Relevant context improved appropriateness, while irrelevant context reduced it and destabilized retrieval in the 320-run experiment.One provider-goal signal also carried products into the final recommendation despite service rules intended to exclude them.Source: §1 · p. 3
  • The study contributes a decision-level account separating customer evidence from provider-side context and introduces the decision context package for selecting what a personalization decision should receive.The authors note that evidence comes from a controlled offline experiment with two constructed customers, one request, and one primary model.Source: §1 · pp. 3–4

2 Conceptual Foundations of Context-Sufficient Personalization

The conceptual framework treats personalization as a decision for one customer and one current request, informed by customer evidence, current intent, and provider-side context. It defines appropriateness by whether a response serves that intent, reflects relevant context, and respects constraints.

2 Conceptual Foundations of Context-Sufficient Personalization

  • Personalization draws on three inputs: customer evidence indicates likely customer value, current intent identifies the immediate objective, and provider-side context establishes what is possible, permitted, or advisable.These inputs are distinguished from neighboring concepts in Table 1.Source: §2 · p. 4, Table 1 · p. 4

2.1 The Personalization Instance

  • The personalization instance is one customer, one current request, and the response generated for that request; context sufficiency is specific to that decision.It is distinct from the customer relationship, which spans interactions and accumulates evidence over time.Source: §2.1 · pp. 4–5

2.2 Customer Evidence and Current Intent

  • Customer evidence indicates what a customer is likely to value, while current intent identifies what the customer is trying to accomplish now.Historical preferences can point away from the current request, so intent provides the reference point for appropriateness.Source: §2.2 · p. 5

2.3 Provider-Side Context

  • Provider-side context is situational information that changes what response is appropriate; its relevance depends on the decision, not merely its availability or source.The framework distinguishes environment, goal, operational, world, and guardrail context, without requiring every instance to use each type.Source: §2.3 · pp. 5–6

2.4 Personalization Appropriateness

  • Personalization appropriateness measures whether a response preserves current intent, reflects relevant provider-side context, and respects applicable constraints.It differs from predicted preference and from recommendation change alone.Source: §2.4 · p. 6

3 A Theory of Context Sufficiency

Context sufficiency is the smallest set of provider-side information that covers what matters to a customer’s current request. The theory explains how context relevance, redundancy, and interaction shape appropriateness, and uses the Context-Sufficiency Frontier to locate that set.

3.1 The Context-Sufficiency Framework

  • Provider-side context spans environment, goal, operational, world, and guardrail information; these are a map of possible sources, not a checklist for every request.Which types matter depends on the interaction and the coverage it requires.Source: §3.1 · pp. 8–9

3.2 Relevance and Coverage

  • Relevance depends on whether a signal bears on the current request, while coverage asks whether the supplied set adequately represents its circumstances; volume alone does not establish either.A relevant signal can still be redundant when the request, customer evidence, or other context already covers the issue.Source: §3.1 · p. 9, §3.2 · pp. 9–10

3.3 Why Generative AI Changes the Problem

  • Generative AI lets providers supply changing context at service time, but the system may apply signals in unintended ways, making irrelevant or conflicting input consequential.Unlike conventional systems, new situational facts need not first be engineered into the recommendation system.Source: §3.3 · pp. 10–11, Figure 3 · p. 10

3.4 Four States

  • The four states are insufficiency (relevant information is missing), sufficiency (the minimum adequate set), saturation (additional information is redundant), and interference (extra context reduces appropriateness).Interference can begin in retrieval: altered candidate pools may leave later ranking stages unable to recover displaced options.Source: §3.4 · pp. 11–13

3.5 Context Interaction

  • Complementary signals can increase one another’s contribution, whereas conflicting signals can reduce contribution and response stability; therefore, context volume alone cannot explain quality.The effect of a signal depends partly on the other information supplied.Source: §3.5 · p. 14

3.6 The Context-Sufficiency Frontier

  • For each context-set size, the Context-Sufficiency Frontier records the highest attainable appropriateness; its rise, plateau, or fall marks insufficiency, saturation, or interference.Sufficiency begins at the smallest size that reaches the highest level, and the frontier is specific to the interaction’s current intent.Source: §3.6 · pp. 15–16, Figure 5 · p. 15

4 Empirical Test of the Theory

The study tests context sufficiency by varying provider-side signals while holding the customer, request, catalog, and generative system constant. It measures appropriateness and stability, traces signals through search and recommendation, and includes a limited cross-model check.

4.1 Study Logic and Setting

  • The retailer’s generative recommender translated customer evidence, the request, and supplied context into catalog search, then ranked candidates into 25 recommendations.This setup made it possible to observe context at both search planning and final recommendation.Source: §4.1 · p. 16

4.2 Experimental Design

  • The full-factorial design tested all 32 combinations of five context signals, repeated five times for each of two constructed customers making the same request.The 320 runs produced 8,000 ranked recommendation positions; ten runs failed validation and were excluded.Source: §4.2 · p. 17

4.3 Context Signals

  • The five signals represented world, goal, and guardrail context; environment and operational context were not manipulated.Two signals were relevant to the request, one was partly redundant, and three did not help answer it.Source: §4.3 · pp. 17–18

4.4 Measures

  • Appropriateness averaged intent preservation, contextual alignment, and constraint compliance; the study also measured repeat-run stability and traced signals through search, candidate pools, and recommendations.The frontier analysis selected context sets on some repetitions and evaluated them on held-out repetitions, with comparisons calibrated to run-to-run variability.Source: §4.4 · p. 18

4.5 Cross-Model Validation

  • A 50-case check using GPT-6 Astra showed the same directional pattern, but did not reproduce the factorial design or independently estimate the frontier.The authors treat this as a robustness check, not a second experiment.Source: §4.5 · p. 19

5 Results

Across the experiment, relevant context improved appropriateness, while irrelevant context could reduce it, destabilize retrieval, or weaken constraint compliance. Smaller context sets outperformed the full set for both customers, although the evidence does not establish a precise universal cutoff or fully test every proposed mechanism.

5 Results

  • Without provider-side context, overall appropriateness was .82, from .99 intent preservation, .48 contextual alignment, and 1.00 constraint compliance.Table 3 summarizes tests of the context-sufficiency propositions.Source: §5 · p. 19, Table 3 · p. 20

5.1 Proposition 1: Relevance

  • Across experimental conditions, each relevant context signal increased appropriateness by .023 on a 0-to-1 scale, while each irrelevant signal reduced it by .034; irrelevant signals were ignored, screened out, or adopted.Adoption of wallpaper context caused the largest decline (-.060), bringing out-of-scope wallpaper and wall-cladding products into final recommendations.Source: §5.1 · pp. 20–21

5.2 Proposition 2: Sufficiency

  • Held-out appropriateness rose from .84 to .94 for Customer 1 and .81 to .92 for Customer 2 with smaller sets, then fell to .88 and .76 with all five signals.The frontier is consistent with a smaller sufficient set, but nearby points fell within run-to-run variability, preventing a precise cutoff or universal composition claim.Source: §5.2 · pp. 22–23, Figure 6 · p. 22

5.3 Proposition 3: Saturation

  • The competitor promotion contributed .091 with no context but generally .017 to .025 when added to sets already containing one to three signals, while the partly redundant mirror signal added relatively little.The result is suggestive of saturation, but redundancy was not independently manipulated while relevance was held constant.Source: §5.3 · p. 23

5.4 Proposition 4: Context Interaction

  • Three interactions emerged: mirror and competitor signals interacted at .026, candle and mirror at -.040, and sofa and wallpaper at +.028.The positive sofa-wallpaper interaction was not predicted; the results support evaluating signals in the context set they enter.Source: §5.4 · p. 23

5.5 Proposition 5: Interference

  • With all five signals, appropriateness fell below the best smaller set for both customers, while mean Jaccard similarity declined from about .23 with no context to about .12–.14 with four to five signals.Interference mainly involved reduced contextual alignment and constraint compliance and unstable retrieval, rather than a substantial loss of intent preservation.Source: §5.5 · p. 24

5.6 Cross-Model Robustness

  • The 50-case GPT-6 Astra check matched the main experiment’s directional pattern but was not a second test of all five propositions or an independent frontier estimate.It is a cross-model robustness check based on a subset of the experiment.Source: §5.6 · pp. 24–25

6 Discussion

The experiment finds that the relevance of provider-side context matters more than its amount, and the discussion argues that generative personalization requires transferring situational judgment through context selection and constraints.

6.1 What the Results Show

  • Relevant context improved appropriateness, while irrelevant context reduced it; adding signals beyond useful context eventually weakened consistency and control.The complete context set was not the best-performing set for either customer.Source: §6.1 · p. 25
  • Additional context altered search and destabilized candidate retrieval, sometimes weakening responses to relevant facts or allowing excluded products into the final list.The customer’s request remained largely intact, so interference did not generally redirect recommendations away from it.Source: §6.1 · p. 25
  • A recommendation can still look broadly suitable while becoming less responsive to the situation, less stable across runs, or less compliant with provider rules.Source: §6.1 · p. 25

6.2 Contributions to Service and Personalization Research

  • Personalized-response appropriateness depends on the provider’s situation at the moment of service, extending service research’s situational logic to the provider side.Provider-side context makes situational knowledge available to the personalization decision.Source: §6.2 · pp. 25–26
  • Situational judgment requires selecting which facts matter for a request and carrying constraints through every stage, not merely transferring facts to the system.The experiment found uneven relevance screening, a search constraint not carried into ranking, and increasingly inconsistent recommendations as context increased.Source: §6.2 · p. 26
  • The Context-Sufficiency Frontier defines the smallest context set that reaches the best attainable appropriateness for a personalization instance.This separates customer evidence, which can grow more informative, from provider context, whose usefulness depends on the present decision.Source: §6.2 · p. 26
  • Generative models screened relevance inconsistently, so providers cannot simply supply all available information and delegate context selection to the model.The model sometimes ignored a signal, spent search effort on another, and adopted a goal despite a scope restriction.Source: §6.2 · p. 26

6.3 Implications for Research on Generative Personalization

  • Recommendation changes alone do not establish context effects; repeated runs and tracing context through search, retrieval, and ranking help distinguish effects from stochastic variation.The paper’s discussion broadens interference to intermediate processing that reduces appropriateness, reliability, or control.Source: §6.3 · p. 27

7 Managerial Implications

The managerial implication is to assemble a decision-specific package of relevant provider context, enforce constraints throughout the process, and monitor more than apparent customer fit.

7 Managerial Implications

  • The decision context package is the smallest set of current provider-side information needed for a particular personalization decision.It is assembled for an interaction rather than fixed as a customer profile.Source: §7 · p. 27
  • Select context by relevance, not availability; include information when it can change what is appropriate for the current request and is not already adequately represented.Source: §7 · p. 27
  • Constraints should be enforced at each service stage because a rule applied during search may not govern later ranking.In the wallpaper case, the exclusion rule was not repeated after search planning, allowing wallpaper products to remain candidates for ranking.Source: §7 · pp. 27–28
  • Applying eligible business priorities only after generating customer-appropriate options is a proposed architecture, not an experimentally established result.Source: §7 · p. 28
  • Monitor appropriateness, consistency across runs, constraint compliance, and where context enters retrieval and ranking, not just whether outputs look relevant.Source: §7 · p. 28
  • Estimate a Context-Sufficiency Frontier by testing plausible signal combinations across representative requests with repeated runs, then use it as a stopping rule.Source: §7 · p. 28

8 Limitations and Future Research

The study establishes context-composition effects under controlled conditions, while its tested outcomes, profiles, signals, and systems limit how far the empirical frontier can be generalized.

8 Limitations and Future Research

  • The study measures personalization appropriateness, not downstream outcomes such as conversion, satisfaction, retention, or profitability.Whether the context set that maximizes appropriateness also maximizes customer or firm value remains open.Source: §8 · p. 28
  • The controlled experiment uses two constructed customer profiles making the same request, so it does not establish how sufficient context sets vary across customers, requests, or service settings.The focal request performed reasonably well without provider context, making observed insufficiency relatively mild; future work should vary intent and examine cases where provider information is essential.Source: §8 · pp. 28–29
  • The empirical frontier uses one open-weights model and five signals representing three of the framework’s five context types, leaving variation across architectures and context forms unresolved.The smaller 50-case GPT-6 Astra test reproduced the directional relevance pattern but did not reconstruct the full frontier.Source: §8 · p. 29
  • Future research should examine context freshness and test whether multi-stage constraints, later introduction of provider goals, or selective context requests improve system design.These extensions address context changes over time and the observed effects of context on retrieval, stability, and constraint compliance.Source: §8 · p. 29
  • The next question is how the sufficient set changes across customers, decisions, systems, and service environments.Source: §8 · p. 29

9 Conclusion

The paper adds provider-side situational knowledge to customer evidence as a requirement for appropriate personalization. Its experiment finds that relevant context helps, whereas irrelevant context can undermine appropriateness, stability, and rule compliance.

9 Conclusion

  • Personalization depends both on what the provider knows the customer may value and on the situation in which the decision is made.Source: §9 · p. 29
  • Relevant context improved personalization, while irrelevant context reduced appropriateness and stability and could alter retrieval or let a provider-goal signal carry excluded products through the process.A 50-case GPT Astra 6 cross-model check produced the same directional relevance pattern as the full experiment.Source: §9 · p. 30
  • The Context-Sufficiency Frontier identifies when relevant circumstances are adequately covered, while the decision context package translates that principle into an interaction-specific unit.The conclusion calls for supplying what the interaction needs, enforcing constraints throughout the process, and transferring provider selection along with situational judgment.Source: §9 · p. 30
Loading 2610.00654v1…