ReLoop: A Self-Correction Continual Learning Loop for Recommender Systems

1 INTRODUCTION

• Conventional retraining uses newly collected feedback as supervision but does not account for errors made by previous model versions.The authors identify this as a missed opportunity to accumulate knowledge from past recommendation experience.Source: §1 · p. 1
• Prior work on distillation and continual learning addresses imitation of previous models and catastrophic forgetting, respectively, but not continual correction of past errors.The paper also situates existing recommender work around feature interactions [4] [24], user behavior modeling [17] [29] [30], and multi-task learning [14].Source: §1 · p. 1
• Figure 1 connects sequential training procedures through a self-correction module that defines errors from model predictions and user feedback labels.The authors expect training to reduce these errors for better online serving.Source: Figure 1 · p. 2
• ReLoop uses sample-wise gaps between prediction scores and ground-truth labels in a customized loss that encourages each new model to reduce prior errors.The framework is presented as generic and applicable to existing recommendation models.Source: §1 · p. 2
• Offline experiments and an online A/B test show consistent improvements over baselines with ReLoop.The evaluations include public benchmark datasets and a private industrial dataset.Source: §1 · p. 2
• The paper presents ReLoop as a generic self-correction framework with a customized-loss module requiring nearly no engineering work in online serving.Its effectiveness and simplicity are evaluated in offline experiments and an online A/B test.Source: §1 · p. 2

2 RELOOP FRAMEWORK

• The framework section introduces ReLoop as a learning paradigm and compares its self-correction module with knowledge distillation.Its described components include the production training loop, error definition, and integration into CTR prediction.Source: §2 · p. 2

2.1 Training Loop In Production

• In production, each training cycle uses logged implicit feedback to train and deploy a model, but adjacent models train independently and ignore prior prediction errors.ReLoop introduces a self-correction module to learn from those previous mistakes.Source: §2.1 · p. 2

2.2 Self-Correction Module

• ReLoop defines sample error as the gap between predicted CTR and the ground-truth label, then penalizes current predictions that perform worse than the previous version.The previous prediction can supply the last error; incorrectly predicted samples can receive emphasis when the current model does not improve.Source: §2.2 · pp. 2–3

2.3 Training Strategy

• ReLoop combines a self-correction loss with binary cross-entropy, with α balancing their contributions to CTR prediction training.The loss is formulated as the sum of both terms.Source: §2.3 · p. 3

2.4 Comparison with Knowledge Distillation

• Knowledge distillation mimics a teacher model's behavior, whereas ReLoop's self-correction module learns from the previous model's errors.For a positive sample, the self-correction penalty assists cross-entropy when the current prediction is below the previous prediction, and adds no extra penalty when it is higher.Source: §2.4 · p. 3

3.1.1 Experiment Setup.

• Experiments use four public datasets and a large-scale industrial dataset, with public data split 8:1:1 for training, validation, and testing.Public-dataset comparisons include LR, FM, DeepFM, and other state-of-the-art baselines.Source: §3.1.1 · p. 3

3.1.2 Experiment results.

• 1.42%, 1.65%, 2.91%, and 4.69% AUC improvement relative to backbone models; ReLoop also outperforms the other baselines across four public datasets.Knowledge distillation is the best baseline on almost all datasets.Source: §3.1.2 · pp. 3–4, Table 1 · p. 4

3.2 Evaluation in Production

• On the production dataset, ReLoop outperforms both the baseline and knowledge distillation on AUC and Logloss, with little training cost and no inference cost.Source: Table 2 · p. 4, §3.2 · p. 4
• 1.46% average CTR improvement over the baseline (A/B - AA) in one-week online A/B tests.The test compared two groups of more than 300 thousand active users each after one week of A/A testing.Source: Table 3 · p. 4, §3.2 · p. 4

• CTR prediction ranks candidate items using fine-grained user, item, and context features, and typically involves large-scale sparse data.Source: §4 · p. 4
• CTR prediction research largely develops model structures, while continuous training-loop optimization receives less attention; ReLoop addresses that gap through self-correction from past errors.Source: §4 · pp. 4–5
• Recommender-system continual learning periodically retrains on the same task with new, potentially shifted data and evaluates performance on new data, unlike the task-sequence focus on catastrophic forgetting.Source: §4 · p. 5
• Self-distillation uses previous model predictions as soft labels, but ReLoop addresses prior-model mistakes rather than assuming the teacher is always correct.Source: §4 · p. 5

5 CONCLUSION AND FUTURE WORK

• ReLoop penalizes samples where the current model performs worse than before, using a customized loss to support self-correction.The framework is described as applicable to any existing recommender model.Source: §5 · p. 5
• Experiments on four public datasets and one large-scale industrial dataset report superiority over other state-of-the-art methods, alongside an effective industrial Newsfeed A/B test.The passage reports these outcomes without giving metric values.Source: §5 · p. 5
• Future directions include finer-grained errors such as list-wise utility or session-based revenue, and self-correction implementations beyond loss.Source: §5 · p. 5