Fixing a Model That Learned Worse Cancer Means Lower Risk: Monotonic Constraints in Bladder Cancer Recurrence Prediction

Saram Abbas; David Thomas; Naeem Soomro; Rishad Shafik; Rakesh Heer; Kabita Adhikari

Source-linked AI summary

Fixing a Model That Learned Worse Cancer Means Lower Risk: Monotonic Constraints in Bladder Cancer Recurrence Prediction

Saram Abbas, David Thomas, Naeem Soomro, Rishad Shafik, Rakesh Heer, Kabita Adhikari

arXiv:2610.00858v1Oct 1, 2026cs.LG

TL;DR

Clinical prediction models can assign lower recurrence risk to more severe disease, a reversal that conventional performance and SHAP checks may miss. This paper tests feature-direction counterfactuals and applies monotonic constraints; the constraints eliminated observed violations without reported performance loss. The evidence comes from one trial with internal cross-validation, and independent external validation is required before clinical use.

Problem
Standard discrimination, calibration, and SHAP checks can miss models that assign lower recurrence risk to more advanced disease.
Source: §1 · p. 2
Method
The framework worsens each directed feature one step at a time to test predicted risk, then applies guideline-derived monotonic constraints where direction violations occur.
Source: §1 · p. 2
Results
47.5% of unconstrained XGBoost counterfactuals were reversed on average across seven directed features, versus 0.0% for the constrained model; constraints removed violations at no cost to performance.
Source: §3.2 · p. 5, §5 · p. 11
Takeaways & Limitations
The authors conclude that a counterfactual direction test belongs in routine validation before clinical model deployment.
Source: §5 · p. 11
The evidence is from one trial of 435 patients with internal cross-validation; independent external validation is required before clinical use.
Source: §4.4 · p. 11

Abstract

from arXiv · show

Background and Objective: Clinicians expect recurrence risk to climb with cancer severity. In a UK multicentre trial, an unconstrained XGBoost model learnt that higher tumour stage and carcinoma in situ predicted lower recurrence risk, and discrimination, calibration, and SHAP were all blind to it. We developed a counterfactual testing framework to detect this inversion and a monotonic-constraint framework to remove it without hurting performance. Methods: BOXIT enrolled 472 patients with protocol-mandated cystoscopy across 51 UK sites (2007-2012); 435 had at least two years' follow-up (153 recurrences, 35.2%). We developed a counterfactual direction test and a monotonic-constraint correction, with constraint directions drawn from the EORTC and EAU risk systems, and evaluated both against unconstrained XGBoost and logistic regression on 18 predictors (seven directed) over 50 cross-validation folds. The test worsened each patient on one directed feature at a time to check whether risk fell; SHAP direction and calibration were also assessed. Key Findings and Limitations: Tumour stage and carcinoma in situ were associated with lower recurrence, opposite to medical intuition; the unconstrained model reversed carcinoma in situ counterfactuals in 90.2% of cases and stage in 74.3%. Discrimination ($\Delta$AUC 0.005, p=0.47), calibration, and SHAP magnitude were all blind to the inversion. Monotonic constraints eliminated every violation at no cost to discrimination (0.723 vs 0.718) and outperformed EORTC (p=8.9e-16). Limitations: single trial, internal-external validation only. Conclusions and Clinical Implications: A model that had learned this inversion passed every conventional check. A counterfactual direction test, run as a single refit with pre-specified monotonic constraints, catches this failure at no cost to performance and should be routine before clinical deployment.

1. Introduction

The paper identifies a blind spot in standard model checks: predictions can reverse clinically expected feature directions while overall performance and SHAP appear acceptable. It introduces a counterfactual direction test and monotonic constraints to detect and correct such reversals.

1. Introduction

The motivation includes risk tables with contemporary concordance of 0.554 and reported overestimation in BCG-treated patients.NMIBC recurrence can trigger repeated cystoscopy, resection, and treatment episodes, with a reported three-year cumulative cost of £8,735 per patient.Source: §1 · p. 2
Higher stage and carcinoma in situ were assigned lower recurrence probabilities than Ta, despite being more advanced disease.Discrimination, calibration, and SHAP assess overall accuracy or feature importance, but do not check whether risk moves in the expected direction.Source: §1 · p. 2
Similar reversals were reported across other cancer and non-urological models, with treatment intensity offered as a consistent possible explanation.The authors note that no established method systematically detects these reversals.Source: §1 · p. 2
The proposed framework tests whether predicted risk moves in the clinically expected direction, then applies monotonic constraints where violations are found.Constraint directions are drawn from guideline-endorsed clinical risk systems.Source: §1 · pp. 2–3

2. Methods

The study used BOXIT trial data and guideline-derived directions to compare constrained and unconstrained models, then tested whether worsening a directed feature lowered predicted recurrence risk. Performance and calibration were assessed through repeated cross-validation.

2.1. Study Design and Data Source

472 patients with intermediate- or high-risk NMIBC were enrolled across 51 UK sites from 2007 to 2012, with protocol-mandated cystoscopic surveillance.Recurrence was assessed on a fixed schedule by local pathologists blinded to later model predictions.Source: §2.1 · p. 3

2.2. Participants and Outcome

435 patients formed the analysis cohort after 37 event-free patients with under two years’ follow-up were excluded; 153 recurrences occurred (35.2%).The two-year floor reflects the EAU-guideline period of closest surveillance for high-risk disease; median follow-up was 4.7 years.Source: §2.2 · pp. 3–4, Figure 1 · p. 3

2.3. Clinical Direction Priors

Constraint directions came from EORTC and EAU risk systems, covering seven features whose higher values cannot lower predicted risk.The past-year recurrence count remained in the model unconstrained; recurrence rate was defined using prior two-year recurrences per year, excluding the registration tumour.Source: §2.3 · p. 4

2.4. Feature Selection and Modelling

Boruta selection yielded 18 predictors, and constrained XGBoost was compared with an otherwise identical unconstrained model and L2-regularised logistic regression.Hyperparameters were tuned on a fixed 80:20 split, then frozen for 50 stratified five-fold cross-validation evaluations.Source: §2.4 · p. 4

2.5. The Counterfactual Direction Test

For each patient and directed feature, the test changed that feature to the next-worse observed value while holding others fixed, then checked whether predicted risk fell.The test was run across all 50 folds for constrained XGBoost, unconstrained XGBoost, and logistic regression; SHAP direction and partial dependence were assessed separately.Source: §2.5 · p. 4

2.6. Statistical Analysis

Discrimination used mean AUC across folds with paired Wilcoxon comparisons, while calibration and parsimony were assessed using pooled out-of-fold predictions.Calibration was evaluated before and after cross-fitted Platt recalibration; the parsimony sweep reduced feature-set size by pooled SHAP importance.Source: §2.6 · pp. 4–5

3. Results

In BOXIT, unconstrained models sometimes assigned lower recurrence risk to worse disease, and conventional checks did not reveal the inversion. Monotonic constraints removed violations without reducing discrimination.

3.1. The cohort’s own biology runs backwards

Observed recurrence fell with worsening stage—45.1% at Ta, 28.4% at T1, and 18.8% at Tis—and the pattern persisted among patients without carcinoma in situ.Among those without carcinoma in situ, recurrence was 47.4% at Ta and 28.3% at T1 (p<0.001).Source: §3.1 · p. 5

3.2. The unconstrained model learned the inversion

Worsening carcinoma in situ lowered unconstrained predicted risk in 90.2% of counterfactuals and raising stage did so in 74.3%; across seven directed features, the mean reversal rate was 47.5%.The counterfactual direction test worsened one feature by one observed level while holding the others fixed; the constrained model had zero violations by construction.Source: §3.2 · p. 5, Table 1 · p. 5
The unconstrained model reversed number of tumours in 19.7% of counterfactuals, although its cohort-level recurrence pattern agreed with clinical teaching.Recurrence rate, another feature whose cohort-level pattern agreed with teaching, was reversed in 7.6%.Source: §3.2 · p. 6

3.3. Nothing in standard validation caught it

Mean AUC was 0.718 for the unconstrained model versus 0.723 for the constrained model (paired difference 0.005, p=0.47), while calibration and SHAP magnitude did not expose the inversion.Cross-fitted Platt recalibration produced a slope of 1.00 with AUC unchanged at 0.718; stage and carcinoma in situ did not enter either fit’s SHAP top ten.Source: §3.3 · p. 6

3.4. Constraints fix it, and cost nothing

The constrained model reduced every detected violation to zero and outperformed EORTC (mean AUC 0.723 versus 0.628; p=8.9×10−16), while matching logistic regression (0.723 versus 0.713; p=0.14).It did so at no cost to discrimination; Figure 3 compares the constrained model, logistic regression, and EORTC.Source: §3.4 · p. 6, Figure 3 · p. 8

3.5. The deployable model can be smaller

The sample of 435 supported roughly 5.4 predictors against a 1,454-patient target for an 18-predictor model, while a five-predictor model achieved 0.727 versus 0.723 for all 18.Only the eight-predictor rung was significantly worse than the best model; performance fell sharply at four and three predictors.Source: §3.5 · p. 8

3.6. The inversion is not explained by the follow-up exclusion

Higher stage had a hazard ratio below 1 in the full cohort (0.59, 95% CI 0.45–0.77), analysis cohort (0.58, 95% CI 0.45–0.76), and audit model (OR 0.51, 95% CI 0.37–0.70).Because the inversion was present before the two-year follow-up exclusion, that exclusion did not explain it.Source: §3.6 · pp. 8, 10

3.4. Constraints fix it, and cost nothing

The constrained model separated recurrence-free survival more than EORTC bands (39.3 versus 17.3 points) and provided 31% higher net benefit at a 0.30 threshold.It also spared 7 more cystoscopies per 100 patients at that threshold.Source: §3.4 · p. 6, Figure 4 · p. 9

4. Discussion

The recurrence inversion persisted across cohort policies and appeared even in a prospective trial with protocol-mandated cystoscopy. Treatment by indication is a plausible but unproven explanation; the authors recommend a directional counterfactual audit alongside calibration.

4.1. Why the inversion shows in the unconstrained model

Treatment by indication may explain the inversion, but the dataset cannot fully separate treatment intensity from disease severity.The inversion persisted under every cohort policy tested, making exclusion the weaker explanation; the authors caution that an unconstrained learner fits associations shaped by clinical decision-making as well as biology.Source: §4.1 · p. 10

4.2. It happened in a clean trial

Despite protocol-mandated cystoscopy, the inversion escaped discrimination, calibration, and SHAP; routine records with less complete follow-up may be at greater risk.Source: §4.2 · p. 10

4.3. Clinicians distrusting these models are not confused

A clinician rejecting a directionally implausible output may be right: feature explanations establish importance, not whether the feature acts in the right direction.The authors recommend a directional counterfactual audit alongside calibration, using one refit against prespecified clinical priors.Source: §4.3 · pp. 10–11

4.4. Limitations

External validation is required before clinical use, and the treatment-by-indication explanation remains unproven because the trial lacks an independent treatment-intensity measure.The EORTC comparison is optimistic because scores used coded trial items rather than published tables; neither inversion nor constraint is a causal claim.Source: §4.4 · p. 11

5. Conclusion

The conclusion is that unconstrained models can learn clinically inverted relationships that conventional checks miss. A counterfactual direction test and monotonic constraints caught and removed the inversion without reducing performance.

The counterfactual direction test is the more durable contribution, and the authors recommend it as routine validation before clinical deployment.In this trial, monotonic constraints removed the inversion at no cost to performance.Source: §5 · p. 11