PaperlayerView exact revision on arXiv

Source-linked AI summary

Rare Disease Classification in Video Capsule Endoscopy

Amit Kumar Patel, Debesh Jha

arXiv:2610.03733v1q-bio.TOeess.IV

TL;DR

Rare VCE findings are difficult to detect because they occupy less than 1% of labeled frames and provide limited signal. HCE combines video-level disease-presence conditioning with per-frame Expected Gain Arbitration on a frozen model. It improves rare-class recall over temporal smoothing and optimal transport in all four evaluation settings, with statistically confirmed GALAR gains over temporal smoothing.

  • Problem

    Rare pathological findings make up less than 1% of VCE frames, and some classes provide little signal for detection.

    Source: §1 · p. 1

  • Method

    HCE combines video-level disease-presence conditioning with Expected Gain Arbitration, a per-frame correction that reassigns a prediction only when expected macro-F1 gain is positive.

    Source: §1 · p. 2, §3.3 · p. 3

  • Results

    HCE improves rare-class recall over temporal smoothing and optimal transport in all four settings; GALAR gains over temporal smoothing are statistically confirmed in both scopes.

    Source: §5 · p. 6, §6 · p. 10

  • Takeaways & Limitations

    The results support training-free, video-conditioned decoding for improving rare-disease recall in capsule-endoscopy triage without retraining the classifier or using test-time labels.

    Source: §6 · p. 10

    Significance testing is underpowered with two cross-validation folds; conditioning becomes inactive for classes whose presence AUC is near chance, and generalization beyond capsule endoscopy is untested.

    Source: §5.3 · p. 9

Abstract

from arXiv · show

Video capsule endoscopy offers a non-invasive way to examine the small bowel, however, the rare but clinically important findings that appear only in less than 1\% of labelled frames, provide very less signal for detection. As a result, while models can achieve high overall performance, they almost completely miss these rare findings. We condition each frame's decision on evidence from the rest of that patient's video, not on the frame's own posterior alone, supplying information a single frozen frame can never contain. Building upon Expected Gain Arbitration, a per-frame, macro-F1-optimal correction over the confusion matrix, we further compose it with video-disease-presence conditional resulting in our decoding method Hierarchical Conditional-Expected-gain (HCE). We evaluate HCE on two independent VCE datasets, Kvasir-Capsule and GALAR, against a static temporal-smoothing baseline and an optimal-transport (OT) frame-level correction. HCE significantly improves rare-class recall over a temporal-smoothing baseline on GALAR (2.7--3.1x; 95\% CI excludes zero in both scopes) and shows a similar, though not statistically confirmed, gain on Kvasir-Capsule (up to 3.4x). Against OT, HCE still improves rare-class recall in all four evaluation settings (+0.016 to +0.029), and additionally improves macro-F1 and MCC.

1 Introduction

Rare VCE findings are clinically important but constitute less than 1% of frames, while existing imbalance approaches face retraining costs or limited per-frame information. HCE combines video-level disease-presence conditioning with a macro-F1-oriented frame correction, and the paper evaluates it on GALAR and Kvasir-Capsule using video-clustered significance tests.

1 Introduction

  • Rare pathological frames make up less than 1% of video frames, and some pathologies are also rare across patients or visually weak.The authors argue that flagging these findings is crucial for clinical usefulness despite high overall classifier performance.Source: §1 · p. 1
  • Existing training-based imbalance methods require retraining, while inference-time corrections can remain limited by per-frame information.The introduction motivates using information beyond an individual frame to provide richer context.Source: §1 · p. 1
  • 1.7–3.4× over temporal smoothing and 1.2–1.9× over optimal transport: HCE improves rare-class recall in all four settings without retraining the frozen backbone.The reported rare-class recall values are 0.064 and 0.101 on Kvasir-Capsule and 0.059 and 0.060 on GALAR.Source: §1 · p. 2
  • On GALAR, conditioning alone helps where video-level presence varies (+0.010, +0.045) but is near-inert where videos contain most classes (+0.003, +0.004).Composed with EGA, it gives the best rare recall in three of four settings and the highest mean across them.Source: §1 · p. 2
  • Five of forty method–metric comparisons resolve statistically, all on GALAR; Kvasir-Capsule intervals are wide and none excludes zero.The study uses per-video cluster bootstrapping because frames within a video are correlated.Source: §1 · p. 2

2 Related Work

The related work situates the paper at the intersection of class-imbalanced learning, inference-time decision correction, and VCE classification. It contrasts retraining-based approaches, post-hoc corrections, robust feature learning, video aggregation, and the selected backbone.

2.1 Class imbalance and long-tailed recognition

  • Class-imbalance approaches such as focal loss, class-balanced loss, and decoupled training address the training objective but require classifier retraining or fine-tuning.The paper contrasts these with decoders that operate at inference time on a frozen model.Source: §2.1 · p. 2

2.2 Post-hoc and plug-in correction

  • Post-hoc logit adjustment and optimal transport correct decisions using priors or distribution matching, while this work conditions correction on video-level disease presence.The paper presents video-level conditioning as addressing the base-rate floor faced by single-frame corrections.Source: §2.2 · p. 2

2.3 Robust feature learning

  • Robust representation-learning methods using frequency-domain or contrastive features are described as insufficient to rescue long-tailed classes.Source: §2.3 · p. 3

2.4 VCE datasets and classification

  • Kvasir-Capsule and GALAR contain correlated frames within patient videos, and the paper adds non-local video-level disease-presence conditioning beyond local temporal smoothing.The datasets contain 117 videos and over 47,000 labeled frames across 14 classes for Kvasir-Capsule, and 80 videos with 3.5 million annotated frames across 29 labels for GALAR.Source: §2.4 · p. 3

2.5 Architecture baselines

  • DenseNet201+Swin-T is selected as the classifier backbone after comparison with CNN, transformer, and other baseline architectures.The chosen dual-stream model combines a DenseNet CNN branch for local features with a Swin Transformer branch for global context.Source: §2.5 · p. 3

3 Methodology

The method modifies frozen classifier decisions at inference using correlated frame evidence within a video. It applies temporal smoothing, per-frame Expected Gain Arbitration, and video-level disease-presence conditioning, then composes the latter two as HCE.

3.1 Setup

  • The decoder acts on frozen frame posteriors at inference time, using video-level disease presence as additional global information for each frame’s local decision.Source: §3.1 · p. 3

3.2 Temporal smoothing (preprocessing)

  • Temporal smoothing averages log-posteriors within a sliding window in each video before any presence conditioning or gain arbitration.HCE includes this preprocessing, so its improvements are on top of temporal smoothing.Source: §3.2 · p. 3

3.3 Expected Gain Arbitration (EGA)

  • EGA reassigns a frame only when the expected macro-F1 gain is positive, using validation-fit class statistics and greedy coordinate ascent.At test time, each frame is scored in a single pass against frozen statistics; the implementation uses exact finite differences of Fβ, with βc=4 for the bottom third of classes by validation prior and 1 elsewhere.Source: §3.3 · pp. 3–4

3.4 Video-level presence conditioning

  • Video-level conditioning estimates disease presence from top-k smoothed frame posteriors using a cross-fitted logistic model, then forms an effective class prior and adds a reliability-weighted log-ratio offset.The reliability weight shrinks toward zero when presence prediction is near chance; conditioning is applied at whole-video rather than sub-video granularity.Source: §3.4 · p. 5

3.5 Hierarchical Conditional-Expected-gain

  • HCE applies EGA to frame posteriors shifted by the video-conditioning offset, combining a per-video prior adjustment with global confusion-matrix arbitration.The paper describes these mechanisms as orthogonal, with EGA serving as the fallback when reliability weighting drives the conditioning offset toward zero.Source: §3.5 · p. 5

4 Datasets & Experiments

Experiments use Kvasir-Capsule and GALAR, with distinct label sets and rare classes. Models and decoding settings are evaluated across folds and seeds, with video-level validation and cluster-bootstrap comparisons.

4 Datasets & Experiments

  • Kvasir-Capsule has 117 videos and 47,238 labeled frames across 14 classes; GALAR has 80 videos and rare classes as scarce as Foreign body (0.0017%) and IBD (0.0004%).Kvasir-Capsule rare classes are Erosion (1.06%), Blood-fresh (0.95%), and Erythema (0.34%); GALAR uses a 12-class video-disjoint pathology subset.Source: §4 · p. 5
  • DenseNet201+Swin-T with focal loss was used for decoding experiments, with 2 folds × 3 seeds per dataset and single-model and 3-seed ensemble evaluations.Training used AdamW and dataset-specific epoch limits; reported results use complete validation and test splits.Source: §4 · p. 5
  • Decoder hyperparameters were selected by leave-video-out cross-validation, and significance was assessed with a per-video cluster bootstrap and practical-significance floors.The bootstrap resamples whole videos because frames within a video are strongly correlated.Source: §4 · p. 5
  • In the illustrated Kvasir-Capsule case, video evidence shifts a missed Erosion frame from Normal 0.61/Erosion 0.39 to Erosion 0.58/Normal 0.42, and arbitration retains the reassignment.Forty-seven frames in the video carry erosion; top-k pooling raises the per-video erosion prior ×24.Source: Figure 2 · p. 6
  • Table 1 reports Kvasir-Capsule decoding results as mean±sd over 3 seeds × 2 folds for single models and 2 folds for ensembles; arrows mark practically meaningful differences with 95% bootstrap CIs excluding zero versus temporal baseline.Bold marks the best value per column within the table block.Source: Table 1 · p. 6

5 Results and Discussion

HCE’s rare-recall gains over temporal smoothing are statistically confirmed on GALAR, while its rare-recall gains over OT are not. Ablations tie conditioning’s contribution to dataset differences and identify a tradeoff between rare recall and overall balance.

5 Results and Discussion

  • 2.7–3.1× higher rare-class recall than temporal smoothing on GALAR is significant in both scopes; Kvasir-Capsule ensemble gains up to 3.4×, with its CI including zero.Table 2 covers GALAR decoding methods and marks comparisons whose per-video bootstrap CIs exclude zero and meet a practical-difference threshold.Source: §5 · p. 6, Table 2 · p. 7
  • Against OT, HCE’s rare-class recall gains are +0.016, +0.019, +0.029, and +0.016 across Kvasir-single, Kvasir-ensemble, GALAR-single, and GALAR-ensemble, with all CIs including zero.HCE significantly improves macro-F1 over OT on GALAR; its MCC gain is significant for single models but not ensembles.Source: §5 · pp. 6–7
  • On GALAR, EGA edges HCE on F1 and MCC by 0.003–0.01, while HCE’s rare-class precision trails OT on Kvasir and temporal smoothing on GALAR.On Kvasir-Capsule, HCE achieves the best macro-F1 and MCC outright.Source: §5 · p. 7

5.1 Ablations

  • Conditioning alone lifts rare recall on Kvasir-Capsule but is near-inert on GALAR; composed with EGA, HCE has the best mean rare recall (0.071) and leads in three of four settings.GALAR videos contain most classes, so video-level presence estimates vary little; on Kvasir-Capsule ensemble, HCE reaches 0.101 versus 0.074 for conditioning alone and 0.071 for EGA alone.Source: §5.1 · p. 8

5.2 Discussion & Analysis

  • HCE dominates every baseline on Kvasir-Capsule in both MCC and rare recall, while on GALAR it trades some MCC for rare-recall gains relative to EGA.The MCC-versus-rare-recall plot compares methods across both datasets and single-model and ensemble scopes.Source: §5.2 · p. 9, Figure 4 · p. 9

5.3 Limitations & Future Work

  • Significance testing is underpowered with two CV folds; most gains are not statistically confirmed, and generalization beyond capsule endoscopy is untested.Reliability weighting approaches zero when cross-fitted presence AUC is near chance, leaving conditioning inactive for the rarest classes and HCE equivalent to EGA on that tail.Source: §5.3 · p. 9
  • The comparison omits a Frank-Wolfe solver for EGA’s target fixed point and per-class validation threshold tuning as a simpler post-hoc alternative.The authors identify both as additional baselines that would sharpen the comparison.Source: §5.3 · pp. 9–10

6 Conclusion

HCE is a training-free, video-conditioned decoder for improving rare-disease recall in capsule-endoscopy triage. Across two datasets it improves rare-class recall over temporal smoothing and optimal transport, with the strongest statistical support on GALAR and a rare-class precision trade-off.

6 Conclusion

  • Across two datasets, HCE improves rare-class recall over temporal smoothing and optimal transport in all four evaluation settings.GALAR gains over temporal smoothing are statistically confirmed in both scopes; Kvasir-Capsule and rare-recall comparisons against optimal transport are directionally consistent, but their intervals include zero.Source: §6 · p. 10
  • GALAR gains over temporal smoothing are statistically confirmed in both scopes, as are gains over optimal transport in macro-F1 and single-model MCC.Comparisons on Kvasir-Capsule and rare-recall comparisons against optimal transport are directionally consistent, but their intervals include zero.Source: §6 · p. 10
  • HCE trades rare-class precision for recall and shows a small F1/MCC drop relative to EGA alone on GALAR.The authors frame HCE as a practical, low-cost way to surface rare-disease frames within a capsule study, not as an autonomous diagnostic decision.Source: §6 · p. 10

7 Appendix

The appendix reports model and training-recipe selection, decoder ablations, imbalance diagnostics, and evaluation details. It retains focal loss with random sampling and whole-video conditioning for the main configuration.

7.1 Baseline selection

  • Nine architectures were trained on Kvasir-Capsule; DenseNet201+Swin-T was selected for macro-F1 and MCC, while DINOv2 ViT-L/14 led in rare-class recall and precision.The comparison used weighted cross-entropy on the official split.Source: §7.1 · p. 11

7.2 Loss and sampler selection

  • Focal loss led macro-F1 and MCC for single models, while weighted cross-entropy led those metrics for ensembles; focal was retained after an OT-alpha check against raw argmax.The follow-up check favored focal on both F1 and MCC.Source: §7.2 · p. 12

7.3 Class-balanced sampler ablation

  • Class-balanced sampling did not reliably improve rare recall: its apparent gains came from 2 of 6 seeds, while single-model HCE macro-F1 fell significantly.The direct per-video bootstrap comparison against focal was not significant (all p ≥0.44), so focal remained the main configuration.Source: §7.3 · p. 12

7.4 Conditioning granularity ablation

  • From whole-video conditioning down to L = 3 frames, macro-F1, MCC, rare recall, and rare precision stayed within noise; whole-video conditioning was chosen for simplicity.The finer granularity was not reported as harmful.Source: §7.4 · p. 12
  • OT’s macro-F1 gain crosses zero as the majority pool grows, from up to +0.07 at small pools to −0.046 beyond roughly 42k–69k frames.Kvasir-Capsule’s natural imbalance lies on the positive side of the crossover, while GALAR’s natural pool lies beyond it.Source: §7.4 · p. 12
  • On Kvasir-Capsule, temporal-window F1 rose from 0.250 at W=0 to 0.286 at W=31; on GALAR, it rose from 0.176 to 0.180 and plateaued near W≈11–15.The GALAR result slightly declined by W=31.Source: §7.4 · p. 12

7.5 Experiment details

  • Kvasir training ran up to 100 epochs; GALAR used 30 epochs, capped majority training classes at 40,000 frames, and evaluated label-blind 10% frame subsamples.The subsampling was applied independently per split and did not change which videos were held out.Source: §7.5 · p. 12
  • Decoder hyperparameters were selected by leave-video-out cross-validation without test-data access, and significance used a per-video cluster bootstrap with practical-significance floors.The floors were 0.005 for F1/MCC and 0.003 for rare recall.Source: §7.5 · p. 12

The macro-F1 plug-in and the base-rate wall

The macro-F1 plug-in view turns the metric’s optimum into a weighted posterior decision rule. Its class-dependent correction also explains why the optimal rule eventually stops boosting an extremely rare class.

The macro-F1 plug-in and the base-rate wall

  • At the macro-F1 optimum, the metric behaves like a weighted linear score, yielding arg max_c a_c p_c(x) with a_c = 2/(P_c + T_c) and offset δ_c = log a_c.The weights are evaluated at their optimal values.Source: The macro-F1 plug-in and the base-rate… · p. 13
  • As prior π_c approaches zero, precision becomes linear in π_c and g_c approaches log π_c plus a constant, canceling the rare-class prior boost.Below the base-rate wall, the macro-F1-optimal rule stops boosting the rare class without a tuned threshold or constant.Source: The macro-F1 plug-in and the base-rate… · p. 13
  • Logit adjustment is the special case g_c = const; under real imbalance, a nonconstant g_c can prevent over-boosting the rare tail and the associated MCC cost.The passage describes g_c as near log 2 for well-separated head classes and strongly negative after a class’s precision collapses.Source: The macro-F1 plug-in and the base-rate… · p. 13

Why EGA has the right form

EGA solves the macro-F1 plug-in fixed point with a stable greedy coordinate-ascent procedure. It reassigns frames only when the expected macro-F1 effect is positive.

Why EGA has the right form

  • Because the plug-in rule is a fixed point and naive iteration diverges, EGA uses greedy coordinate ascent on the exact expected macro-F1 effect of frame reassignments.Counts are estimated on validation under the base decoder and iterated twice to reflect earlier reassignments.Source: Why EGA has the right form · p. 13
  • With a calibrated posterior, EGA moves a frame to the best alternative class only when the expected macro-F1 effect is positive.This is the decision rule used to make the fixed-point solver stable.Source: Why EGA has the right form · p. 13
  • Like Schultheis et al. (NeurIPS 2023), EGA uses coordinate ascent, but solves a simpler two-pass greedy fixed point within a video-conditioned per-frame pipeline.The authors adopt this solver because it composes with their video-level conditioning offset.Source: Why EGA has the right form · p. 13

Escaping the wall: video-level conditioning

Frame-level prevalence factorizes into disease prevalence across patients and finding prevalence within affected videos, so video-level evidence can condition a frame’s decision. HCE boosts a class when video-level presence is supported and suppresses it otherwise.

Escaping the wall: video-level conditioning

  • Frame-level prevalence factorizes into the probability a patient has class c and the probability a frame shows c given that the patient has it.The two factors are estimated at different levels; their product is conserved, but the classification difficulty differs because precision depends on prevalence and false-positive rate.Source: Escaping the wall: video-level… · p. 13
  • HCE conditions the class offset on video v, using r_c estimated from validation videos containing c and q_c(v) from a top-k aggregate of frame posteriors.The resulting decoder falls outside the plug-in theorem’s family of decoders with a single global offset.Source: Escaping the wall: video-level… · pp. 13–14
  • As q_c(v) approaches 1, the offset gives a positive boost; as it approaches 0, it suppresses class c in that video.The parameters α and λ are selected on validation macro-F1; the video-conditioned offset is intended to protect precision compared with uniform rare-class boosting.Source: Escaping the wall: video-level… · p. 14

Optimal transport (Sinkhorn) baseline

The OT baseline uses Sinkhorn’s algorithm to assign frames according to a transport plan between frame posteriors and a target class marginal. Its assignment adjusts global predicted class proportions, with an unbalanced variant relaxing the class constraint.

Optimal transport (Sinkhorn) baseline

  • The OT baseline solves an entropic optimal-transport assignment with Sinkhorn’s algorithm using per-frame log-posteriors and a target class marginal.The target marginal is the empirical class prior raised to a tuned power α.Source: Optimal transport (Sinkhorn) baseline · p. 14
  • Frames are assigned by row-argmax under the transport plan, re-matching predicted class proportions to a chosen target without changing within-frame posterior rank order.The plan uses a uniform row marginal and temperature-scaled log-posteriors.Source: Optimal transport (Sinkhorn) baseline · p. 14
  • The unbalanced variant relaxes the column constraint by softness κ, with hard OT at κ = 1 and plain argmax at κ = 0.This avoids forcing a confidently normal frame to satisfy a rare-class quota unsupported by its evidence.Source: Optimal transport (Sinkhorn) baseline · p. 14

Temporal smoothing

Temporal smoothing averages posteriors within each video, while HCE’s rescue patterns differ between Kvasir-Capsule and GALAR. Whole-video bootstrap intervals confirm selected GALAR gains, but do not resolve rare-recall comparisons against OT.

Temporal smoothing

  • Temporal smoothing averages each posterior across a within-video neighborhood, suppressing isolated artifact spikes while preserving contiguous lesion runs.Under the temporal+OT decoder, Kvasir-Capsule F1 rises from 0.250 at W=0 to 0.286 at W=31, while GALAR rises from 0.176 to 0.180 and plateaus around W≈11–15.Source: Temporal smoothing · p. 14, Figure 6 · p. 17

7.7 Rescue confusion matrices

  • On Kvasir-Capsule, HCE raises Angiectasia and Lymphangiectasia recall by +0.308 and +0.115, while several GALAR classes lose recall as predictions shift toward normal.Kvasir’s largest gains recover frames previously assigned to Normal mucosa; GALAR’s normal recall rises only +0.016.Source: §7.7 · p. 14, Figure 7 · p. 19, Figure 8 · p. 20

Complete Bootstrap Intervals

  • On GALAR, rare-recall gains over temporal smoothing resolve in both settings (+0.037 single-model; +0.040 ensemble), while OT comparisons resolve for macro F1 (+0.022, +0.025) and single-model MCC (+0.094).No rare-recall comparison against OT resolves; rare precision against temporal smoothing has negative point estimates (−0.022, −0.042), but neither interval excludes zero.Source: Complete Bootstrap Intervals · pp. 14–15, Table 9 · p. 18
Loading 2610.03733v1…