Source-linked AI summary
Can Large Language Models Identify Meaningful Touchpoints in Conversion Attribution?
Jinqi Wu, Sishuo Chen, Zhangming Chan, Yong Bai, Chao Yi, Han Zhu, Shuodian Yu, Lei Zhang, Sheng Chen, Chenghuan Hou, Jian Xu, Chaoyou Fu
TL;DR
Conversion-attribution heuristics can miss semantically relevant touchpoints, especially implicit associations that matter for identifying conversion contributors. The paper builds and annotates SILVA, evaluates LLMs with different prompting strategies and model families, and uses LLM-attributed labels for CVR training. LLMs uncover substantial implicit relevance, with pairwise prompting and larger models performing better, while LOTUS improves offline CVR performance.
Problem
Heuristic touchpoint selection can miss implicitly-related interactions that are semantically relevant but lack explicit taxonomy or collaborative-filtering signals.
Method
The paper constructs the SILVA benchmark, systematically evaluates LLM touchpoint selection across prompting strategies and model scales, and uses LLM-attributed labels for CVR training.
Results
LLMs identify explicit touchpoints effectively and discover substantial implicit relevance, while pairwise prompting and larger models perform better; LOTUS gains 0.35 pp GAUC over Base and 0.15 pp over CABB.
Takeaways & Limitations
LLM-based semantic reasoning offers a supported direction for touchpoint selection and can improve production-scale CVR prediction when its attributed labels are integrated.
Abstract
from arXiv · showhide
Touchpoint selection in conversion attribution, namely identifying meaningful touchpoints contributing to conversions, is essential for e-commerce recommendation and online advertising. Current selection methods rely heavily on collaborative-filtering-based heuristics, which fail to align with user-perceived semantic intent. Through human annotation, we reveal a significant semantic gap: many implicitly-related, semantically relevant touchpoints remain undetected by existing rules. Therefore, we systematically evaluate the capability of Large Language Models (LLMs) in identifying these hidden associations. Our evaluation shows that while LLMs effectively uncover a substantial portion of implicitly-related touchpoints, significant room for improvement remains in their selection performance. Furthermore, we analyze the impact of different prompting strategies and foundation model choices on identification performance, providing valuable insights into their reasoning patterns and effectiveness. These insights offer a new roadmap for transitioning conversion attribution from mechanical rule-matching to human-aligned semantic reasoning. Moreover, we leverage the LLM-attributed conversion labels for enhancing industrial CVR model training and achieve significant offline performance gains, showing the potential of LLMs in conversion attribution.
1 Introduction
Conversion attribution depends on both selecting meaningful touchpoints and allocating credit across them, but heuristic selection can miss semantically related interactions. The paper evaluates LLMs for this gap and applies LLM-attributed labels to CVR prediction.
- Conversion attribution allocates credits across past touchpoints and supports advertiser reporting, CVR label generation, and platform revenue.
- Heuristic selection based on product taxonomy and collaborative-filtering signals can miss implicitly-related touchpoints lacking explicit taxonomy or engagement overlap.
- The study systematically evaluates representative LLMs on SILVA across model scales and prompting strategies for explicit and implicit touchpoint identification.
- LLMs identify explicit touchpoints well and uncover many implicit ones, but substantial room for improvement remains in implicit touchpoint selection.
- Pairwise reasoning outperforms listwise reasoning, with the gap narrowing as model capability increases.
- Proprietary flagship LLMs only slightly outperform open-weight counterparts, while smaller models below 100B parameters largely lag behind.
- LOTUS improves main-goal CVR prediction by 0.35 percentage points over Base and 0.15 pp over CABB in a large-scale e-commerce setting.
2 Related Work
Prior conversion-attribution research emphasizes credit allocation and CVR training while generally overlooking touchpoint selection beyond the consumed product’s item or shop.
- Prior studies focus on allocating weights among touchpoints through causal modeling and using attribution results to train CVR prediction models.
- Most existing approaches consider only touchpoints under the same item or shop as the consumed product, leaving cross-item relevance under-addressed.
3 The SILVA Benchmark
SILVA defines touchpoint labels and distinguishes explicit rule-based associations from implicit semantic associations. It contains 1,000 conversion events and 69,680 enriched touchpoints annotated from a large-scale e-commerce platform.
- Touchpoint selection assigns each preceding interaction an Irrelevant, Explicit, or Implicit label for a given conversion.
- Explicit relevance includes taxonomy matching and collaborative-filtering similarity based on metadata or statistical associations.
- Implicit relevance captures semantic associations such as functional complementarity and scenario or audience alignment that explicit rules miss.
- SILVA contains 1,000 conversion events and 69,680 preceding touchpoints within a 3-day lookback window.
- Each SILVA touchpoint includes multimodal information about the clicked product, including its image, title, shop, and category metadata.
- Human annotation finds that explicitly-related touchpoints comprise 19.09% and implicitly-related touchpoints 14.91% of the dataset.
4 Harnessing LLMs for Touchpoint Selection
The study evaluates eight LLMs and two prompting protocols for selecting explicit and implicit touchpoints on SILVA. LLMs identify explicit relations well, but implicit selection remains difficult; pairwise prompting and larger models generally perform better.
- Main findings: Pairwise prompting reaches 0.8979 explicit F1 but only 0.4929 implicit F1, while listwise prompting reaches 0.8372 and 0.4608, respectively.The best implicit F1 is 0.5426, below the best explicit F1 of 0.9095.
- Prompting protocols: Pairwise prompting averages 0.6954 Overall F1, versus 0.6490 for listwise prompting across models.The authors attribute this advantage to candidate-level judgments and report that listwise prompting tends to over-filter touchpoints.
- Foundation models: GLM-5.1 and DeepSeek-V4-Pro achieve 0.7086 and 0.7056 Overall F1 with pairwise prompting, close to GPT-5.5 at 0.7197.Smaller Qwen variants lag behind flagship models, particularly on implicitly-related touchpoints.
- Prompting protocols: The pairwise–listwise gap is 1.3 pp for Gemini-3.1-Pro and 3.0 pp for GPT-5.5, but 6.2 pp and 11.9 pp for the smaller Qwen variants.Figure 4 indicates that pairwise gains are larger when listwise Overall F1 is lower.
- Ablation study: Thinking improves Overall F1 by +0.0196 under listwise prompting and +0.0082 under pairwise prompting on average.The authors describe these gains as marginal and variable across models and protocols.
5 Application: Enhancing CVR Prediction
The authors apply LLM-selected touchpoints as positive samples for an auxiliary indirect-conversion task in a production-scale CVR model. LOTUS improves offline GAUC over both an in-shop baseline and a heuristic cross-shop method.
- CVR prediction: LOTUS achieves absolute offline GAUC gains of 0.35 pp over Base and 0.15 pp over CABB for the main conversion goal.Base is the in-shop attribution baseline, while CABB is a heuristic method covering explicit cross-shop touchpoints.
- CVR prediction: The 0.35 pp improvement over Base demonstrates the value of incorporating cross-shop touchpoints into CVR training.The experiment uses selected touchpoints as positive samples for an auxiliary indirect conversion prediction task.
- CVR prediction: The additional 0.15 pp gain over CABB suggests that LLM semantic reasoning identifies implicit relations beyond explicit heuristic rules.The authors frame this result as evidence of incremental semantic signals in touchpoint selection.
6 Conclusion
The paper identifies a semantic gap in heuristic conversion attribution and evaluates LLMs as a way to recover implicitly-related touchpoints. Integrating those selections into CVR prediction improves GAUC over both production and heuristic baselines.
- Conclusion: Implicitly-related and explicitly-related touchpoints comprise 14.91% and 19.09% of all annotated touchpoints in SILVA.The benchmark supports analysis of touchpoints overlooked by traditional heuristics.
- Conclusion: LLMs can bridge part of the semantic gap by reasoning over product metadata and user-behavior context, although implicit selection remains challenging.The conclusion also reports that pairwise reasoning and larger foundation models improve performance.
- Conclusion: LLM-identified touchpoints improve GAUC by 0.35 pp over the production baseline and 0.15 pp over the heuristic CABB competitor.The production baseline considers only in-shop touchpoints.