PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal Stratified Allocation for Rare-Event Onset Forecasting in Dependent Sequences

Jaskaran Singh

arXiv:2609.04420v1math.STq-fin.STstat.MLstat.TH

TL;DR

For rare-event learning on dependent sequences, the paper asks how to allocate a limited sample across classes and how to compare scores across retraining periods. It derives finite-population allocation results and tests them on explosive-regime onset forecasting; the predicted design ordering holds, but the predicted rarity-margin relationship does not.

  • Problem

    The paper asks how to allocate a fixed training subset across rare and abundant classes and how to compare scores from models fitted at different times, where pooled rankings may not preserve within-block rankings.

    Source: §1 · p. 2

  • Method

    The paper derives finite-population variance and optimal allocation results for class-weighted stratified sampling, then tests the predictions across sampling designs.

    Source: §1 · p. 3, §8 · p. 12

  • Results

    The predicted ordering of the four constructions holds, but the predicted dependence of the performance margin on rarity does not.

    Source: §14 · p. 21

  • Takeaways & Limitations

    The allocation ordering is reproduced empirically, while the rarity-margin prediction marks a limit of the design-based account for a procedure that is not itself design-based.

    Source: §14 · p. 21

    The allocation theory assumes equal unweighted loss dispersion across classes, leaves the mixing coefficient unestimated, and treats negative rows as independent despite their autocorrelation.

    Source: §13 · p. 19

Abstract

from arXiv · show

Let a finite population of n labelled examples carry a class-weighted loss, with pi*n in a rare positive class weighted by N0/N1. We study estimation of total risk from a subsample K << n under designs allocating K0 and K1 draws to the two strata. We derive the exact finite-population variance of the weighted risk estimator under class-conditional sampling without replacement and solve for the optimal allocation. The class multiplier inflates positive-stratum dispersion by the imbalance ratio, causing that ratio to cancel from the optimal allocation and making equal, rather than proportional, allocation the natural default. Simple random sampling is dominated by an explicit between-stratum term; an exact bias identity shows that cluster-representative selection has no general unbiasedness guarantee; and a Serfling bound transfers the allocation result to selection error over a finite candidate set. Under the implemented truncation, the realised allocation ratio is gamma=min(2*pi/f,1), independent of n, yielding the parameter-free efficiency prediction A(pi,f)=gamma/[pi(1-pi)(1+gamma)^2]. A separate measurability result bounds the record occupied by a labelled example, determining the required train-test separation and controlling departure from block independence under absolute regularity. We test these predictions on forecasting the onset of statistically explosive price regimes, dated ex post by the Phillips-Shi-Yu procedure, using 350 U.S. equities from 2004-2011 with under 1% positive rows and five purged forward blocks. The predicted ordering of the four designs holds, and at the 10-day horizon the five design points are ordered exactly as predicted by A (Spearman rho=1, exact p=0.0167). The predicted dependence on pi across horizons does not hold; we identify the channels lying outside the design-based argument.

1 Introduction

The introduction frames rare-class allocation and cross-period score comparability as finite-population problems, then applies allocation theory to forecasting the onset of statistically explosive price regimes. It also outlines structural results and practical evaluation contributions.

2 Background: when does a price count as explosive?

  • The PSY procedure scans price data for trajectories too accelerated to fit a random walk and produces retrospective flagged and unflagged dates.Its output is a statistical-test rejection, not a diagnosis of economic reality.Source: §1 · p. 2, §2 · pp. 3–4

1 Introduction

  • The forecasting task ranks stocks not currently flagged by the probability of a new flagged episode within 5, 10, or 20 trading days, using information available by day t.The PSY procedure supplies retrospective labels; examples inside episodes are excluded, label windows begin after feature dates, and blocks are purged.Source: §1 · pp. 2–3
  • The allocation theory derives exact finite-population variances and makes equal class counts optimal under the default positive-class multiplier, with relative efficiency 1/{4π(1 −π)} over proportional allocation.It also identifies a between-class variance term for uniform sampling, places clustering-based representative selection outside the framework by an exact bias identity, and uses a Serfling bound for split-selection error.Source: §1 · p. 3
  • A measurability lemma bounds how much data a labelled example can touch, deriving the purge length and bounding departure from block independence under a mixing assumption.A rank-invariance result says the validation-relative score map leaves within-period ranking metrics unchanged, so pooled gains measure misalignment between retraining dates.Source: §1 · p. 3
  • The introduction proposes alert-budget evaluation reporting detection rate, warning lead, and alert burden, and documents a test-period ranking inversion with a validation-time gating rule.A label-conditional sign reversal in a feature cluster flags the inversion when a covariate-shift monitor does not.Source: §1 · p. 3
  • A validation-relative percentile score makes rankings from models retrained at different dates comparable without test labels and roughly quadruples pooled cross-period average precision.The map cannot change within-period ranking metrics.Source: §1 · p. 3
  • The predicted ordering of four designs holds: class-balanced sampling improves average precision over full-data training at every 10-day retention level, while the predicted margin dependence on rarity does not hold.The introduction says the paper reports three channels outside the design-based argument that could account for the failed rarity prediction.Source: §1 · p. 3

3 Preliminaries and notation

  • The notation section distinguishes data, detector, and learning-task symbols from sampling-design, evaluation, and alert symbols; Section 6 uses the latter panel.The symbol key also specifies conventions for indices, horizons, validation quantities, and full-data runs.Source: Table 1 · p. 4, §3 · p. 4

4 Related work

This section situates prospective rare-event forecasting among related methods, defines its onset target, and develops theory for temporal separation, subset allocation, and cross-period score alignment; it also explains that average precision and ROC-AUC can move in opposite directions across horizons because only average precision depends on prevalence.

4 Related work

  • Prior work applies explosive-root tests operationally and pairs them with machine learning for S&P 500 and metal prices; this study adds prospective exclusion, rare-event allocation comparisons, and alert-budget analysis.Option-price deep-learning work uses a different bubble definition; the test family’s sensitivity to permanent volatility shifts is treated as a label threat.Source: §4 · pp. 4–5

5 Problem formulation

  • The evaluation targets future episode onsets rather than ongoing episodes, but filtered eligibility is knowable only ex post and changes the scored population.Filtering produces a disagreement bounded below by 1,573 rows (0.22% of the panel and 5.7% of raw-flagged rows); incomplete horizons are censored rather than labelled negative.Source: §5 · pp. 5–6

6.1 The information span of a labelled example

  • The label can reach h+∆ dates ahead while test features reach back w dates, so a 87-day bound supports separation; absolute-regularity dependence remains assumed and its degradation is unquantified.With w=60, merge gap 2, minimum episode duration 5, and h≤20, the design uses a 140-trading-day gap; non-overlap alone does not establish independence.Source: §6.1 · p. 6

6.2 Subset construction as a sampling design

  • Class-stratified and class-balanced probability samples are unbiased at fixed model state, while class stratification can beat uniform sampling through reduced allocation noise; K-means representatives lack a general unbiasedness guarantee.The representative bias vanishes for all model states only if loss is constant within every cluster, which feature-based clustering cannot ensure as boosting changes the loss.Source: §6.2 · p. 7

6.3 Optimal allocation, and what the class multiplier does to it

  • Under class-weighted loss, the optimal split depends on within-class dispersions rather than prevalence, and equal allocation is optimal when unweighted dispersions are equal.The imbalance multiplier cancels from the optimal allocation; proportional allocation instead requires equal weighted dispersions.Source: §6.3 · pp. 7–8

6.4 What the theory predicts, and where it should fail

  • Ignoring the floor at 25, truncated allocation with K*=⌈fn⌉ and N1=πn has a realized ratio depending only on π and f, not n; Section 11.4 finds P1 and P3 supported and P2 contradicted.The paper offers three candidate explanations: the variance bound concerns objective rather than generalisation error, temporally adjacent near-duplicate negatives overstate independent information, and roughly half the h=10 advantage arises in period F2 with an inverted ranking.Source: §6.4 · p. 9

6.5 Score alignment across retraining windows

  • At h=5, pooled average precision rises from 0.0049 to 0.0195 after within-fold rank mapping, although every fold’s internal ordering is unchanged.Thus the pooled gain reflects alignment across separately fitted models, not changes in within-fold classifier ranking.Source: §6.5 · pp. 9–10
  • Average precision rises while ROC-AUC falls across horizons without contradiction: average precision depends on prevalence, and enrichment ratios 5.8, 3.1, and 2.3 decline with ROC-AUC.Average precision should be read against prevalence; only its ratio to prevalence is comparable across horizons.Source: §6.5 · p. 10

7 Data

The study uses a fixed panel of 350 U.S. securities from 2004–2011 and forecasts episodes dated by a recursive right-tailed test. Predictors use information available by day t, while the archive’s undocumented delisting coverage limits interpretation.

7 Data

  • The fixed universe contains 350 securities and 705,061 security-day rows from 2 January 2004 to 30 December 2011, with 1,285 filtered episodes.Securities were selected using pre-sample data; the resulting panel is unbalanced, with no rows imputed across dates.Source: §7 · p. 10
  • Detector thresholds use 2,000 simulated Gaussian random walks per sequence length, with the 95th percentile set separately at each endpoint.The ADF regression uses an intercept and zero augmentation lags; the minimum window follows the stated PSY rule.Source: §7 · p. 10
  • The 17 close-of-day predictors include return, momentum, volatility, volume, moving-average, drawdown, skewness, and market-aggregate features, all using rows no later than t.Ret 20 and momentum 20 are algebraically identical, leaving 16 effective features; preprocessing bounds are estimated on training rows only.Source: §7 · p. 10, Table 3 · p. 11
  • The archive’s undocumented coverage of securities delisted before its snapshot may tilt the universe toward milder episodes, with the direction of the resulting effect unclear.The authors recommend auditing against a delisting-inclusive reference before using the results for capital decisions.Source: §7 · p. 11

8 Evaluation protocol

Evaluation uses five chronological, non-overlapping test periods with purged expanding-window folds. The protocol separates within-period ranking from pooled cross-period score comparability and accounts for dependence in uncertainty estimates.

8 Evaluation protocol

  • Five right-aligned folds use expanding training blocks, 140-date purges, and 126-date validation and test blocks, yielding non-overlapping test periods.The test periods run from July 2009 through December 2011.Source: §8 · p. 11
  • The 140-day purge exceeds the derived 87-day minimum, which includes the 60-day feature window, 20-day horizon, and roughly seven-day episode-confirmation term.Validation data set calibration, score rescaling, and alert thresholds; test labels remain untouched until final evaluation.Source: §8 · p. 11
  • Macro average precision averages fold-level test AP equally, while pooled AP evaluates whether separately trained models’ scores compare across periods.AP is read against prevalence, and the two AP summaries differ because AP is nonlinear.Source: §8 · p. 11
  • Pooled row metrics use 500 moving-block bootstrap draws over whole dates, while episode metrics resample security paths within folds.For subset comparisons, paired intervals resample folds and seed-level differences; the reported sign test, not the descriptive positive-draw fraction, supports inference across configurations.Source: §8 · pp. 11–12

9 Models and scores

The evaluation pairs a prevalence baseline and single-feature baselines with logistic regression and LightGBM. A validation-relative percentile transformation supplies a common within-period rank scale for comparing models trained at different dates.

9 Models and scores

  • The baselines include prevalence, momentum 60, and realized volatility 60; logistic regression and LightGBM provide learned benchmarks under the stated preprocessing and chronology.The prevalence baseline has AP equal to prevalence and ROC-AUC 0.5; the classifiers use balanced class weights.Source: §9 · p. 12
  • Validation-block sigmoid calibration sets a fold-local score scale for threshold selection but does not make scores comparable across folds.The sigmoid is fitted only when validation contains both classes and at least ten positives, then applied unchanged to test scores.Source: §9 · p. 12, §9.1 · p. 12

9.1 A validation-relative percentile score

  • The validation-relative percentile maps a score to its position in that model’s immediately preceding validation-score distribution, without test labels or future data.A score of 0.90 means it exceeds 90% of that model’s validation scores.Source: §9.1 · p. 12
  • The transformation preserves every within-fold ranking metric, so its effect on pooled performance measures misalignment between separately fitted models.The paper uses the uncorrected map for rank-based metrics; it is a relative risk rank, not a probability or conformal p-value with a coverage guarantee.Source: §9.1 · pp. 12–13

10 Training-set reduction

The reduction experiments compare weighted sampling designs under a fixed subset budget, with class-balanced allocation motivated by the weighted-risk variance result. K-means representatives fall outside the unbiased allocation framework and incur substantial construction cost.

10 Training-set reduction

  • At 1%, 2.5%, 5%, 10%, and 20% retained fractions, the study compares uniform, class-stratified, class-balanced, and K-means constructions with row weights.K-means is evaluated only at 1%, 2.5%, and 5%.Source: §10 · p. 13

10.1 Why class-balanced sampling is not a second reweighting

  • Class-balanced sampling uses equal class counts subject to availability and minimums; the class multiplier N0/N1 makes equal allocation the Neyman optimum, not a duplicate correction.The sampling weights estimate the same objective without bias across the class-conditional designs, so unbiasedness alone does not distinguish them.Source: §10.1 · p. 13, §10 · p. 13
  • The allocation theory governs variance at a fixed model state, while boosting changes candidate splits, node statistics, histograms, and tree topology.The paper does not extend its finite-candidate result along the boosting path; near-duplicate negatives and hyperparameter fit remain explanations not separated by the evidence.Source: §10.1 · pp. 13–14

10.2 The cost of K-means representatives

  • K-means representatives have no general unbiasedness guarantee, and the measured end-to-end speedup falls from 0.609× at 1% to 0.044× at 5%.The authors attribute the slowdown to the implementation’s clustering and nearest-centroid costs, not necessarily to representative selection as such.Source: §10 · p. 13, §10.2 · p. 14
  • The reported runtime includes subset construction, fitting, calibration, and test inference, so a speedup below one means the reduced method is slower overall.The experiment runs 885 configurations across three horizons and five folds.Source: §10.2 · p. 14

11 Results

Class-balanced sampling improves average precision over full-data training across tested 10-day retention levels, while validation-relative percentiles improve pooled cross-period scores without changing within-period rankings. Results also reveal an inverted-ranking fold, horizon-dependent limits to the allocation prediction, and alert outcomes under validation-set false-alarm budgets.

11.1 Forward onset ranking

  • Full-data LightGBM has the highest macro average precision at every horizon: 0.0213, 0.0217, and 0.0328 at h=5, 10, and 20.The values are 5.8, 3.1, and 2.3 times the respective prevalence baselines; at h=10 its gap over logistic regression is small.Source: §11.1 · p. 14, Table 4 · p. 15

11.2 One period in which the ranking inverts

  • At h=10, F2 ROC-AUC is 0.137, far below the other folds’ 0.483–0.868 range, while its mean feature-distribution shift does not flag the inversion.The shift measure compares median feature locations and can miss changes in tails, inter-feature correlations, or sector composition.Source: §11.2 · pp. 14–15
  • At F2, four volatility-and-asymmetry features reverse, but the same cluster also reverses at h=5 without failure, and the strongest individual predictors do not reverse.The authors identify ensemble interaction between stable momentum and locally inverted volatility signals as an unresolved multivariate question.Source: §11.2 · p. 15

11.3 Making scores comparable across retraining periods

  • Validation-relative percentiles raise pooled AP to 0.0195, 0.0264, and 0.0326 across horizons, exceeding alternative score scales; at h=5 the gain is roughly fourfold.Within-fold rankings remain unchanged, so the pooled difference reflects cross-period score misalignment rather than the classifier.Source: §11.3 · pp. 15–16, Table 5 · p. 16

11.4 Data reduction: a consistent 10-day pattern, and what five periods can support

  • At h=10, predicted and measured retention-fraction rankings coincide (ρS=1, exact p=0.0167), but predicted rarity dependence across horizons fails (ρS=0.11, p=0.70).At h=20 agreement is partial; at h=5 measured differences are indistinguishable from zero.Source: §11.4 · p. 16
  • At h=10, class-balanced sampling beats full-data AP by +0.0052 to +0.0086 at every fraction, with 2.42× to 3.13× speedups; this is a pattern, not a confirmed effect.The five-fold effective unit is the forward period, and the exploratory search across 54 configurations further limits evidential interpretation.Source: §11.4 · pp. 16–17

11.5 Early warning under a fixed false-alarm budget

  • At h=10, the 2.5% class-balanced model detects 11.5%, 24.1%, and 42.5% of episodes across budgets, with median leads of 5.4, 5.9, and 6.5 days.Realised burdens are 0.61, 1.03, and 1.64 false clusters per security-year; whether the trade-off is worthwhile depends on user costs.Source: §11.5 · p. 18, Table 8 · p. 18

11.6 Robustness

  • Increasing the purge from 140 to 180 trading days preserves percentile-score performance and class-balanced competitiveness at h=10; raising dmin from five to ten preserves patterns where positives remain sufficient.The paper retains five days as the primary setting because it supports the full five-fold chronology.Source: §11.6 · p. 18

12 Discussion

The discussion distinguishes relative improvement from absolute screening performance and interprets the horizons as different rarity–signal settings. It treats the class-balanced model’s advantage in a failing period as a hypothesis about robustness, not a demonstrated mechanism.

12 Discussion

  • AP 0.03 against prevalence 0.007 is a large relative but small absolute improvement; the enrichment ratios are 5.8, 3.1, and 2.3.Only AP/π is comparable across horizons; the enrichment ratios fall in step with ROC-AUC.Source: §12 · p. 19
  • At 5 days, full-data ROC-AUC exceeds 0.85 in four of five folds; at 20 days, the longer window admits more unrelated variation.The 10-day horizon is where a signal exists and the within-horizon prediction P3 is reproduced exactly.Source: §12 · p. 19
  • Roughly half the class-balanced advantage comes from one period where full-data ranking inverts; robustness to that failure remains a hypothesis, not a demonstrated mechanism.The proposed explanation is that denser positive examples may help resist a reversal in the relationship between volatility features and outcome.Source: §12 · p. 19

13 Limitations and future work

The paper bounds its empirical claims by five forward periods and identifies limits in the allocation theory, detector labels, comparisons, and sample scope. It also outlines follow-up tests, including longer panels and checks of horizon coherence.

13 Limitations and future work

  • Five forward test periods bound inference: unanimous signs give p = 0.0625, while the retention-fraction rank test is conditional on the theory’s predicted ordering.The authors say a longer panel, rather than a different statistic, is the remedy for the other claims.Source: §13 · p. 19
  • The allocation theory fixes model state, reaches split selection only at one node, assumes equal class-loss dispersion, and leaves mixing degradation qualitative.Negative-row autocorrelation may lower the effective sample size and reduce the predicted advantage; the relevant dispersion and mixing quantities were not checked or estimated.Source: §13 · p. 19
  • Labels use critical values simulated under homoskedastic Gaussian walks, so volatility shifts may distort the detector’s size and contribute to apparent forecastability.The authors identify relabelling with a wild-bootstrap specification and rerunning headline models as the most valuable unperformed experiment.Source: §13 · pp. 19–20
  • Absent comparators include class-conditional LightGBM bagging, gradient-based one-side sampling, and a temporally thinned subset for estimating effective negative sample size.Refitting with the class multiplier fixed at one would separate dataset-size effects from hyperparameter fit.Source: §13 · p. 20
  • The target is a retrospectively filtered episode onset, not necessarily the first day a live detector would flag an episode.Episode membership requires a two-day merge gap and five-day minimum duration; a raw-transition target is suggested as a robustness check.Source: §13 · p. 20
  • Excluding detector state isolates detector-independent forecasting but understates what an operational system could use.The proposed benchmark adds the detector statistic and its distance to the threshold as predictors.Source: §13 · p. 20
  • Independent horizon models can violate monotonicity because the three targets are nested views of one waiting time; a discrete-time hazard model would enforce coherence.The proposed hazard formulation models the probability of onset at each step conditional on no earlier onset.Source: §13 · p. 20
  • The study covers established, relatively liquid pre-2004 securities, leaves a proposed gating check unvalidated, and does not run its configured detector and universe sweep.The panel excludes subsequent listings; the authors say extending it would test post-2011 market structure and supply power for the sign test.Source: §13 · p. 20

14 Conclusion

The conclusion reports above-prevalence onset ranking and theoretical results about allocation and cross-period score scales. The predicted design ordering appears in the empirical results, but five periods do not confirm the proposed robustness explanation.

14 Conclusion

  • Across all three horizons, the gradient-boosted-tree model ranks onset-adjacent observations above a constant-prevalence score, most consistently at five days.The forecasting protocol excludes observations inside episodes, starts labels after the feature date, and purges blocks according to the separation bound.Source: §14 · p. 21
  • The allocation result makes equal counts and class reweighting complementary, with equal-over-proportional efficiency 1/{4π(1 −π)}; uniform sampling incurs a between-class term, and clustering lacks a general unbiasedness guarantee.Separately, ranking scores within each model’s validation distribution leaves within-period order unchanged; its fourfold pooled average-precision rise measures cross-period score-scale drift.Source: §14 · p. 21
  • The predicted ordering of four constructions holds, and the five retention fractions follow the predicted order exactly within the 10-day horizon.The conclusion says five periods cannot confirm the general claim: unanimous agreement across five paired observations gives p = 0.0625.Source: §14 · p. 21
  • The class-balanced advantage is concentrated partly in one period with an inverted full-data ranking, a proposed robustness hypothesis that a longer panel should test.The authors report the inverted period rather than averaging it away across periods.Source: §14 · p. 21

Code and data availability

The repository contains the manuscript source, generated tables, orchestration scripts, and run artifacts for the primary experiment.

Code and data availability

  • The repository includes primary-run artifacts for 885 configurations across 350 securities, 17 feature columns, three horizons, and five forward folds.Artifacts include paired comparisons, fold diagnostics, cross-period score comparisons, robustness outputs, and per-configuration timing.Source: Code and data availability · p. 21

A Proofs

The proofs establish allocation and selection-error results, characterize temporal separation for features and labels, and identify limits on cluster selection and prevalence-sensitive metrics.

A Proofs

  • Features use a 60-day lookback, while labels can depend on dates through t+h+∆, requiring train-test separation so their date ranges do not overlap.The feature at test date t_te reads no earlier than t_te−w; a training label at t_tr reads no later than t_tr+h+∆.Source: A Proofs · p. 22
  • Under absolute regularity, block dependence is bounded using a gap of g−w−h−∆ and Berbee’s coupling.The validation and test blocks can be coupled to an independent test-block copy with failure probability controlled by β(g−w−h−∆).Source: A Proofs · pp. 22–24
  • K-means representatives have no general unbiasedness guarantee because the bias identity need not vanish when loss varies within a cluster.The clustering depends on x, whereas the loss can depend on Φ.Source: A Proofs · p. 23
  • The optimal class allocation is K_c ∝ N_cS_c, and under the stated weighted-loss condition σ_0 = σ_1 it reduces to equal allocation.The imbalance multiplier cancels from the optimal allocation ratio.Source: A Proofs · p. 23
  • Serfling’s inequality and a union bound over M candidates yield a finite-candidate selection-error bound from the classwise sampling bounds.The classwise tail bound applies to bounded populations sampled without replacement.Source: A Proofs · p. 23
  • A strictly increasing score transformation preserves test-row ordering and thus average precision and ROC-AUC; ties introduced by validation gaps can change them through tie handling.The proof assumes the transformation is injective on test scores for exact ordering preservation.Source: A Proofs · p. 23
  • ROC-AUC is prevalence-free, whereas average precision depends on prevalence, so only AP/π is comparable across horizons under fixed class-conditional score distributions.For a constant score, average precision equals π.Source: A Proofs · p. 24

B Complete fold results

The section presents full-data LightGBM results by horizon and forward fold, pooled-score uncertainty, and remaining subset-method grids.

B Complete fold results

  • Table 9 reports full-data LightGBM results by horizon and forward fold, with Shift defined as test-minus-training feature-median shift in training-IQR units.Shift is the mean absolute feature-median shift.Source: Table 9 · p. 24

C Pooled-score uncertainty

  • Table 10 reports moving-block bootstrap uncertainty for concatenated fold-local calibrated scores.Source: Table 10 · p. 24

D Remaining subset-method grids

  • Table 11 reports subset-minus-full-data average precision by horizon and retained fraction, with K-means slower than full-data training at every setting.End-to-end speedups are 1.84–2.72× for uniform, 2.03–2.89× for class-stratified, and 0.038–0.463× for K-means.Source: Table 11 · p. 24
Loading 2609.04420v1…