Source-linked AI summary
FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applications
Dwipam Katariya, Snehita Varma, Akshat Shreemali, Benjamin Wu, Kalanand Mishra, Pranab Mohanty
TL;DR
Financial Services recommendation must model long-range, heterogeneous interactions across channels and coordinate personalization across interrelated products. FinTRec unifies transformer-based sequential modeling with product adaptation, and the reported results show stronger performance than tree-based baselines.
Problem
Financial Services recommendation must account for long-range behavior across digital and physical channels and interrelated products, while prior sequential work had not explored dynamic non-digital signals such as transactions and payments.
Source: §1 · p. 1Method
FinTRec is a unified transformer framework that uses raw sequential context and a proprietary foundation model, with product adaptation through fine-tuning.
Source: §1 · p. 2, §3.1 · p. 3, §3.3 · p. 4Results
Takeaways & Limitations
Product fine-tuning supports cross-product signal sharing while reducing infrastructure overhead and improving codebase standardization.
Source: §4 · p. 6, §8 · p. 7The pCTR and pCVR models remain in separate codebases, motivating future work on scalable architectures and unification.
Source: §8 · p. 7
Abstract
from arXiv · showhide
Transformer-based architectures are widely adopted in sequential recommendation systems, yet their application in Financial Services (FS) presents distinct practical and modeling challenges for real-time recommendation. These include:a) long-range user interactions (implicit and explicit) spanning both digital and physical channels generating temporally heterogeneous context, b) the presence of multiple interrelated products require coordinated models to support varied ad placements and personalized feeds, while balancing competing business goals. We propose FinTRec, a transformer-based framework that addresses these challenges and its operational objectives in FS. While tree-based models have traditionally been preferred in FS due to their explainability and alignment with regulatory requirements, our study demonstrate that FinTRec offers a viable and effective shift toward transformer-based architectures. Through historic simulation and live A/B test correlations, we show FinTRec consistently outperforms the production-grade tree-based baseline. The unified architecture, when fine-tuned for product adaptation, enables cross-product signal sharing, reduces training cost and technical debt, while improving offline performance across all products. To our knowledge, this is the first comprehensive study of unified sequential recommendation modeling in FS that addresses both technical and business considerations.
1 INTRODUCTION
Financial Services recommendation must account for long-range, heterogeneous behavior across digital and physical channels and coordinate personalization across interrelated products. FinTRec is proposed as a unified transformer framework to address these demands and balance business objectives with user experience.
1 INTRODUCTION
- Prior sequential recommendation work had not explored dynamic non-digital signals such as transactions and payments, despite their dominance in Financial Services.These signals join heterogeneous, multi-channel behavior that matters for real-time personalization at scale under low-latency constraints.Source: §1 · p. 1
- Financial Services serves varied content across interrelated products, so optimizing only for CTR risks clickbait while optimizing only for CVR may degrade user experience.Feed-style personalization may globally rank locally personalized ads, while siloed product models increase maintenance cost and miss cross-product interactions.Source: §1 · p. 1
- FinTRec is a unified transformer framework for personalized advertising and servicing that uses raw sequential context, a multi-objective function, and a proprietary foundation model.Product fine-tuning is described as enabling cross-product awareness while reducing technical debt and deployment complexity.Source: §1 · p. 2
2 TASK DEFINITION
The task ranks a candidate item for a user at a given time using click and conversion signals. Its final score supports balancing engagement, conversion value, and stakeholder-defined urgency.
2 TASK DEFINITION
- The task considers a candidate item shown to a user at time t and models click and conversion probabilities in the financial-services setting.Conversion may occur without a click or platform exposure, including through external sources.Source: §2 · p. 2
- CTR reflects short-term engagement, while CVR estimates conversion probability given the item and user context.The text cautions that optimizing only for CTR can promote low-quality or clickbait content with limited business impact.Source: §2 · p. 2
- The final Ranking Score weights calibrated CTR, predicted CVR times present value, and an urgency score to balance long-term value with short-term business priorities.Urgency is a soft override set at the campaign-owner level, retaining user-preference signals.Source: §2 · p. 2
3 METHODOLOGY
The methodology prepares temporally ordered, tokenized user histories with static, dynamic, and FM context, then models click and conversion behavior with separate transformer architectures. It also adapts a unified model to new products and evaluates it on temporally held-out data using task-specific metrics.
3.1 Data Pre-processing
- FinTRec preserves chronological interaction histories and uses flexible conversion attribution, while the Random Forest baseline flattens histories into snapshots and reduces 768-dimensional FM embeddings to 32 dimensions with PCA.Continuous features are standardized, categorical features are encoded or tokenized, and each interaction is paired with static, dynamic, and FM context vectors.Source: §3.1 · p. 3, Figure 2 · p. 3
3.2 Model Architecture
- The pCTR model uses a causal decoder and clicked-item next-item loss, whereas pCVR uses a bidirectional encoder and pooled sequence representations.pCTR retains static and FM context across sequence time points; pCVR uses context from the latest interaction.Source: §3.2 · pp. 3–4
3.3 Product Adaptation FT
- Product adaptation pretrains a decoder across existing products, then fine-tunes for new products with LoRA and learnable new-token embeddings while freezing the backbone.Linear probing updates only the final dense layer, while full fine-tuning updates all model parameters; all strategies replace the output adapter.Source: §3.3 · pp. 4–5
3.4 Experimental Setup
- Evaluation uses 90 training days and seven days each for validation and testing, with log loss for pCVR and ranking recall@top-k for pCTR.Product adaptation is tested with leave-product-out validation on feed- and placement-style products, with inference aligned to production settings.Source: §3.4 · p. 5
- The pCTR and pCVR models use different transformer architectures, and pCTR training excludes non-clicked steps from its final loss.Figure 5 distinguishes the encoder-only pCVR model from the causal decoder-only pCTR model.Source: Figure 5 · p. 5
4 OFFLINE RESULTS
FinTRec outperforms production tree-based baselines on PGC marketing pCVR, while product adaptation improves pCTR across products. Fine-tuning strategy matters: full fine-tuning leads, while LoRA-FT offers a performance and training-cost balance.
4 OFFLINE RESULTS
- FinTRec’s pCVR is 0.0439 versus production RF at 0.0984 and RF + FM embeddings at 0.0938 on PGC marketing.Table 3 reports the pCVR comparison for PGC marketing content.Source: §4 · pp. 5–6, Table 3 · p. 6
- F-FT achieves up to +26.85% Recall@1, while LoRA-FT reaches +24.21% Recall@1 with <5% parameter updates.These product-adaptation lifts are reported over the product-specific baseline.Source: §4 · p. 6, Table 4 · p. 6
- Removing temporal encodings or FM embeddings degrades pCVR performance, while larger context windows improve accuracy.The results point to the importance of temporal dynamics, financial context, and long-range dependencies.Source: §4 · p. 6
- FinTRec without fine-tuning underperforms in most cases, associated with cross-product pCTR differences and missing product-specific temporal alignment.LP-FT offers smaller gains with <1% parameter fine-tuning, whereas LoRA-FT balances performance and training cost while improving codebase standardization.Source: §4 · p. 6
5 VISIT ABLATION
The section examines visit-level attribution for explaining recommendations, using attention weights and Grad-SAM. It reports that a few touch points can carry predictive power comparable to the full interaction sequence.
5 VISIT ABLATION
- A few touch points can have predictive power comparable to the full interaction sequence.The authors connect this finding to the usefulness of explainable insight in regulatory findings.Source: §5 · p. 6
- Visit-level attribution provides temporal granularity for tracing which behavior sequence led to a recommendation, supporting compliance review.The passage names Fair Lending, Fair Housing, Model Risk Management, and consumer-facing explainability as motivations.Source: §5 · p. 6
- Attention weights are averaged across heads and layers and normalized into a visit-importance distribution; Grad-SAM combines attention weights with their gradients.The methods provide distinct visit-level relevance signals.Source: §5 · p. 6
- Applicability and differences from Integrated Gradients and Gradient-SHAP have yet to be studied on this data.The authors identify this as an open scope boundary for the explainability analysis.Source: §5 · p. 6
6 INFERENCE AND SERVING
The serving setup combines nightly and incrementally updated features with eligible content at request time to produce final rankings. It operates under a strict 120 ms 99th-percentile latency requirement at under 1,500 API requests per second.
6 INFERENCE AND SERVING
- The serving system targets end-to-end latency of 120 ms at the 99th percentile for under 1,500 API requests per second, regardless of model-call count.The deployment uses internally developed platforms for feature and model serving.Source: §6 · p. 6
- Nightly-batched FM embeddings and contextual features are combined with features generated since the last batch job.At request time, retrieved features are joined with content eligible for the user before ranking.Source: §6 · p. 6
- The model ranks eligible content using the retrieved features to generate the final recommendation.This describes the request-time serving sequence.Source: §6 · p. 6
7 ONLINE A/B EXPERIMENTS
Offline simulations compare expected business outcomes and show FinTRec with the largest reported log-loss reduction and a projected simulated PV gain, while actual PV impact remains pending. The evaluation also relates log loss to engagement and uses guardrail and stakeholder-defined metrics.
7 ONLINE A/B EXPERIMENTS
- Lower log loss generally correlates with higher user engagement, but the relationship is not strictly proportional.The simulation compares top-ranked items from the current and new models using the final ranking objective.Source: §7 · p. 7, Table 5 · p. 7
- The simulation explores ranking-weight coefficients to produce a sensitivity curve showing the trade-off between estimated clicks and PV.Product owners select final coefficients to align with business objectives, typically matching historically observed CTR and noting the resulting PV increase.Source: §7 · p. 7, Figure 6 · p. 7
- FinTRec achieves the largest log-loss reduction (-55.38%) with a projected simulated PV gain up to +41.50%; actual PV impact is pending.The authors conclude that offline results support deployment, while noting that simulation does not guarantee a proportional PV lift.Source: §7 · p. 7, Table 5 · p. 7
- Monitoring includes fraud rate, post-session two-hour call rate, on-time payments, and stakeholder-defined metrics such as impression thresholds for content categories.These metrics are used to detect unintended model consequences and content cannibalization.Source: §7 · p. 7
8 CONCLUSION
FinTRec outperforms the traditional tree-based baseline and combines long-range and implicit signals with cross-product fine-tuning. The authors report production-readiness evidence while identifying remaining work on freshness, unified click and conversion models, and explainability.
8 CONCLUSION
- FinTRec shows superior performance over the traditional tree-based baseline.Historical A/B tests, simulations, and an explainability study support the authors’ assessment of production readiness.Source: §8 · p. 7
- Time-aware long-range sequences, implicit feedback, and FM embeddings support real-time inference without costly history retrieval.The FM embeddings draw on years of transactional and other offline context.Source: §8 · p. 7
- Product fine-tuning shares cross-product knowledge and improves offline performance, while separate pCTR and pCVR codebases, nightly-batched embeddings, and incomplete use-case-specific explainability remain limitations.Future work targets scalable unified click and conversion models, same-day interaction awareness without added latency, and finer regulatory alignment.Source: §8 · p. 7
9 AUTHOR BIO
The authors are Capital One data scientists and leaders associated with its recommendation and personalization team. Benjamin Wu previously worked at Capital One before joining NVIDIA.
9 AUTHOR BIO
- Katariya, Varma, and Shreemali are Capital One data science managers and part of the recommendation and personalization team.Their listed locations are McLean, San Francisco, and New York, respectively.Source: §9 · p. 8
- Mishra is a Capital One director of data science, and Mohanty is a senior director of applied research.Their listed locations are San Jose and Seattle, respectively.Source: §9 · p. 8
- Wu previously worked as a Capital One data science manager before joining NVIDIA as a senior solutions architect.Source: §9 · p. 8
A RELATED WORK
Prior sequential recommendation research addresses session behavior and feedback, but the cited work leaves gaps in financial-services coverage, cross-channel context, low-latency representation use, and unified modeling across interrelated products. FinTRec is proposed to address these gaps in financial-services personalization and ad targeting.
A RELATED WORK
- Prior work found transformers effective for session-aware recommendation, but industrial deployment raises scalability and inference-latency challenges, while financial-services transformer applications remain limited.Financial-services interactions also span digital and physical channels, unlike digital-only settings.Source: A RELATED WORK · p. 9
- Existing feedback models leave gaps in jointly capturing positive and exposure signals, sequence truncation, spatial page context, or cross-channel implicit behavior.RACP models exposure-click interactions and intra-page and inter-page context, while TEM4CTR jointly extracts interests from clicked and unclicked sequences.Source: A RELATED WORK · p. 10
- Embedding studies often represent individual streams, while combining heterogeneous financial-services streams with sequential data for low-latency personalization remains understudied.Tabular transformer studies cited here lacked dominant payment and transaction interaction signals.Source: A RELATED WORK · p. 10
- Tree-based financial-services models align with explainability needs but require extensive feature engineering, and product-specific models create fragmented infrastructure and technical debt.Siloed models also miss cross-product interactions, while naive signal merging can improve one task and degrade others.Source: A RELATED WORK · p. 10
- FinTRec is proposed to unify financial-services personalization and ad targeting across interrelated products with heterogeneous sequential signals.The cited prior unification work primarily concerns e-commerce, travel, and digital media rather than financial-services enterprises.Source: A RELATED WORK · p. 10
B CUSTOMER JOURNEY
Financial-services customer journeys combine digital and offline interactions across channels and products. Their granular, timestamped events provide context for sequential modeling, while outcomes may extend beyond direct digital-ad exposure.
B CUSTOMER JOURNEY
- A month-long illustrative journey includes seven touch-points: four digital, one physical-location visit, one transaction, and one external source.The external source example is Google Search.Source: B CUSTOMER JOURNEY · p. 10
- Product acquisitions may follow a broader interaction landscape rather than being attributable to digital ads alone.Users may engage through physical locations and call-center agents as well as digital platforms.Source: B CUSTOMER JOURNEY · p. 10
- Channel, page type, placement, and layout provide context across diverse products, while timestamped events stored across disparate databases can be aligned on a user timeline.Ads may appear across many pages and placements, including welcome modules, account details, and marketing messages.Source: B CUSTOMER JOURNEY · p. 10