PaperlayerView exact revision on arXiv

Source-linked AI summary

Concentration, Not Uncertainty: Why Targeted Synthetic Data Doesn't Help Camouflaged Object Detection

Akshat Dobhal, Sanjay Singh

arXiv:2610.09807v1cs.CV

TL;DR

Camouflaged-object detection needs costly pixel-level masks, and it is unclear how to direct synthetic generation under a fixed budget. The paper tests uncertainty-guided allocation with clustered real images and retraining, finding that concentration rather than uncertainty-based aim explains the training-set effects; it also reports substantial CHAMELEON contamination.

  • Problem

    The paper asks how to allocate a fixed synthetic-data budget across real-image regions for camouflaged-object detection, where pixel-accurate masks are costly to annotate.

    Source: §1 · p. 1

  • Method

    The authors cluster unlabeled real images, score cluster uncertainty, allocate synthetic generation toward high-uncertainty clusters, and retrain the model.

    Source: §2 · pp. 3–4

  • Results

    Same-shape controls attribute observed training-set differences to budget concentration rather than uncertainty direction, while 50 of 76 CHAMELEON images are training-set copies missed by exact hashing.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Under this fixed-budget setup, concentration—not uncertainty-based targeting—accounts for the observed training-set effects.

    Source: §4 · p. 10

    This build’s prospects are bounded by a fixed optimization budget that displaces real-image training, an exhausted foreground supply, a generator no wider than its input, and a mixed target pool.

    Source: §4 · p. 11

Abstract

from arXiv · show

Camouflaged object detection requires pixel-accurate masks, but obtaining such annotations is slow and costly, making synthetic training images an attractive alternative. Under a fixed generation budget, however, it remains unclear which real-image regions to target for synthetic data generation. We study an uncertainty-guided generation strategy that clusters the unlabelled real images, identifies clusters on which the model is least certain, allocates synthetic generation toward those clusters, and iteratively retrains the model. Across 103 training runs, uncertainty-based targeting does not outperform random allocation. Five independent controls further show that this null result is not an artifact: targeted training sets are measurably different from random sets, but the difference is explained by concentrating the generation budget rather than by where uncertainty is concentrated, as every concentration rule we test reproduces the effect and, on boundary accuracy, so does aiming at the clusters the model was most certain about. Separately, we find substantial data contamination in CHAMELEON, with 50 of its 76 images duplicated from training data despite the standard overlap check reporting zero overlap. Together, these results show that, under a fixed synthetic-data budget, budget concentration, not uncertainty-based targeting, accounts for the observed training-set effects.

1 Introduction

Camouflaged-object masks are costly to annotate, motivating synthetic data and the question of how to allocate a fixed generation budget. The paper finds that concentrating the budget, rather than directing it toward uncertainty, accounts for the observed training-set effects.

1 Introduction

  • Pixel-accurate camouflage masks are costly because annotators must locate boundaries that the objects’ appearance is designed to conceal.Synthetic generation paints a fresh background around a real cutout, yielding a labeled pair without annotation.Source: §1 · p. 1
  • The proposed loop clusters unlabeled real images, scores uncertainty, directs synthetic generation toward high-uncertainty regions, and retrains; its components draw on established methods [27].The authors identify the composition of these components as their contribution and report that it did not function.Source: §1 · p. 1
  • Targeted sets differ measurably from random sets, but same-shape controls attribute the difference to concentration rather than where uncertainty points.The control scrambles the uncertainty signal while preserving the allocation shape; the targeted method does not survive it.Source: §1 · p. 2
  • The paper reports a bounded null for uncertainty direction, defended by five controls, and finds 50 of 76 CHAMELEON images duplicated from training data despite zero overlap by exact hashing.The re-saved copies evade byte-level matching; the authors release a re-encode-tolerant detector, confirmed pairs, and a clean protocol.Source: §1 · p. 2

1.1 Related Work

  • The study asks whether acquisition policies can improve synthetic-image selection under a fixed budget, distinct from constructing labeled data in the first place.The task is camouflaged object detection, which seeks pixel-accurate outlines of objects that blend into their surroundings [10].Source: §1.1 · pp. 2–3
  • The paper builds on LAKE-RED and S2R-COD [24], adding an allocation layer while leaving generation, budget, and training unchanged.It frames uncertainty-guided generation as a controlled test of combining standard components, including uncertainty sampling, student–teacher disagreement, and clustering [27] [30].Source: §1.1 · p. 3

2 Methods

The method allocates a fixed synthetic-image budget among clusters of unlabeled real images according to model uncertainty, then retrains within a fixed two-round pipeline. Its comparison arms separate allocation direction from concentration while sharing the base pool.

2 Methods

  • The two-round pipeline trains a student with a slowly updated teacher, appends agreed target images labeled by teacher predictions, then retrains from scratch.The generator renders its pool offline once, and the round count is fixed rather than a convergence criterion.Source: §2 · p. 3
  • Uncertainty-guided allocation clusters target images using k-means over normalized self-supervised embeddings, scores student–teacher disagreement, and fills quotas with foreground cutouts near funded centroids.Cluster scores are averaged over members, and a softmax with temperature τ = 1.0 determines the budget split; the score’s dominant term compares gradient magnitudes.Source: §2 · p. 4
  • A2, B, and C10 append 1,000 images to the shared base pool; A2 uses real photographs, B random renders, and C10 the targeted allocation.The C10-versus-B comparison changes the selected images, while Figure 1 distinguishes allocation concentration from where it points.Source: §2 · p. 4, Figure 1 · p. 5
  • The C-family controls hold allocation concentration fixed while varying direction, enabling comparison of aim without changing allocation shape.CSHUF permutes cluster scores, CINV reverses their rank, and CORACLE uses true endpoint error; Table 1 describes the arms.Source: §2 · p. 4, Table 1 · p. 4
  • A gap counts as a real effect only when it exceeds twice the pooled within-arm Sα standard deviation and every seed agrees on its sign.Smaller gaps are within noise; a gap above the threshold without sign agreement is inconclusive, with verdicts applied per architecture.Source: §2 · p. 5

3 Results

Uncertainty-guided targeting did not beat random allocation, while controls indicate that concentration—not the direction of targeting—accounts for observed training-set effects. The section also identifies weak cluster-level explanatory power and substantial CHAMELEON overlap with training data.

3.1 Does Targeting Beat Random

  • At eight seeds, targeting’s primary-endpoint gap over random was +0.0024 and did not clear the registered bar; four control campaigns likewise left the decisive gap unchanged.The seed expansion found a larger run-to-run spread and a smaller gap than the three-seed estimate; the oracle and unpinned-schedule comparisons also remained within noise.Source: §3.1 · pp. 5–6, Table 2 · p. 6

3.2 Concentration, Not Targeting

  • The probe’s separation survived shuffled or arbitrary targeting and collapsed only when concentration was removed, indicating that it detected allocation concentration rather than aim.Across twenty cells, targeting added +0.0073 Cohen’s d over its own shuffle, while its contrast with arbitrary-cluster targeting was −0.0649.Source: §3.2 · p. 7, Figure 2 · p. 8
  • With allocation shape fixed, reversing the uncertainty signal produced the highest mean Sα in all four cells, but every direction contrast remained within noise.On the post-hoc SINet boundary metrics, both C10−B and anti-targeted CINV−B cleared their bars, while direction contrasts stayed within noise; SINet-v2 resolved none of these effects.Source: §3.2 · p. 7, Table 3 · p. 8

3.3 Why It Fails

  • Clusters were weakly separated, and only about one fifth of endpoint structural-error variance lay between clusters, limiting what cluster-wise allocation could address.The allocation also sent 51.5% of its budget to CAMO despite its 24.8% share of the target pool; the uncertainty score tracked per-pixel error more strongly than structural error.Source: §3.3 · pp. 8–9

3.4 CHAMELEON Is Mostly Training Data

  • Fifty of CHAMELEON’s 76 images (65.8%) matched training-pool photographs despite zero overlap under file hashing.The count combines 41 same-dimension re-encodes and nine geometrically adjudicated matches; the authors advise reporting COD10K-test and NC4K rather than CHAMELEON’s 10-image uncontaminated subset.Source: §3.4 · pp. 9–10, Figure 3 · p. 10

4 Discussion

The discussion separates a positive geometric finding—concentration explains the training-set separation—from a downstream null bounded by the study’s sensitivity. It also identifies pre-allocation checks, a discriminating control, and limits on what this setup establishes.

4 Discussion

  • Concentration, not the uncertainty signal’s direction, produces the separation that makes targeted selection look effective; the downstream null holds only within stated sensitivity.The authors caution that the null does not show uncertainty guidance cannot work, only that the study names the effect size it could detect.Source: §4 · p. 10
  • The CHAMELEON leakage path remained in the target pool: 44 shared images persisted after source-side removal, and detection required a re-encode-tolerant check.The authors attribute no intent and say the source-side removal shows awareness of leakage.Source: §4 · p. 10
  • Three checks requiring no training should precede cluster-wise allocation: test whether data cluster, characterize funded clusters, and locate endpoint error between versus within clusters.The discriminating control preserves the allocation shape while destroying the signal, because random selection differs from targeting in both aim and concentration.Source: §4 · p. 11
  • A real-versus-synthetic AUC near 1.0 is near-vacuous evidence: a lossy re-save of identical images separates at 0.9928.The discussion also notes that a null endpoint result does not establish a null on the quantity the acquisition signal was designed to move.Source: §4 · p. 11
  • The tested setup would need a growing optimization budget, unexhausted foreground supply, a generator wider than its input, and a target pool not separable into signal-distinguishable components.The authors say this does not rule out closed-loop generation generally; it limits what this implementation could establish.Source: §4 · p. 11

5 Conclusion

Under a fixed synthetic-data budget, uncertainty-guided targeting did not beat random allocation; the observed training-set effect came from concentration rather than aim. The conclusion also reports substantial CHAMELEON contamination missed by exact hashing.

5 Conclusion

  • Across 103 trained runs and five controls, uncertainty-guided targeting did not outperform random allocation under a fixed synthetic-data budget.Targeted training sets differed measurably, but concentration rather than aim explained the difference; every tested concentration rule reproduced the effect.Source: §5 · p. 11
  • 50 of CHAMELEON’s 76 images were altered copies of COD10K and CAMO training images, which exact hashing missed.Source: §5 · p. 11
  • The authors recommend testing acquisition rules with controls that could have failed and auditing duplicates beyond exact matches.Source: §5 · p. 11

A Additional Related Work

The related work situates the study among COD benchmarks and detectors, synthetic-data and adaptation methods, and uncertainty-guided allocation approaches. It also frames benchmark integrity and controls that distinguish budget aim from concentration as relevant context.

Additional Related Work

  • COD evaluation uses COD10K [9], CAMO, NC4K [25], and CHAMELEON; reported measures include S-measure [6], E-measure [7], and weighted Fβ.SINet and SINet-v2 [9] are used off-the-shelf, while SegMaR represents architectural progress outside the allocation experiment.Source: A Additional Related Work · pp. 14–15
  • LAKE-RED uses the compositional idea of copy-paste augmentation [13], whereas CyCADA [14] aligns feature distributions; the reported null is bounded to LAKE-RED.Latent diffusion [31] and DDIM [36] provide the generator and deterministic sampling schedule inherited by the pipeline.Source: A Additional Related Work · p. 15
  • DINOv2 [27] and CLIP [30] provide frozen clustering representations, while disagreement scores decide which clusters receive more of the budget.The paper’s same-shape shuffle is described as a direct control for the uncertainty-and-diversity entanglement addressed jointly by BADGE [1].Source: A Additional Related Work · p. 15
  • Prior work on shared dataset bias, cross-split near-duplicates [2], and language-model evaluation contamination motivates explicit benchmark auditing.The paper also cites reproducibility guidance, disconfirming-result publication [17], and cautions about encoding-sensitive comparison probes [15] [28].Source: A Additional Related Work · p. 15
  • Table 4 compares budget-steering systems on whether they run a signal-destroying or same-shape control, which separates aiming effects from concentration effects.The authors present this as a missing control in the literature, not a superiority claim; their own entry records a null.Source: A Additional Related Work · p. 15

B Detailed Method Specification

The detailed specification records training settings, allocation sweeps, metrics, campaign rules, and implementation controls. It also documents data provenance, optimization-budget choices, and the comparison dimensions used to situate budget-steering systems.

B Detailed Method Specification

  • Training uses SINet for 40 epochs and SINet-v2 [9] for 100, with Adam at 10−4, specified batch sizes, 352×352 inputs, and an EMA teacher at momentum 0.996.The configured consistency weights are a = 0.9 and b = 0.3; training uses saliency-weighted BCE, while the acquisition score uses the unweighted form.Source: B Detailed Method Specification · pp. 15–16
  • Table 4 compares systems by acquisition signal, generator control, diversity control, closed-loop feedback, and whether a signal-destroying or same-shape control is run.The last column distinguishes aiming effects from concentration effects; the paper’s own entry records a null rather than superiority.Source: Table 4 · p. 16
  • The four-arm campaign held its base pool byte-identical across 24 runs, with both rounds using the unfiltered 4040-image target set and shared trainer and checkpoint settings.Steps per epoch were pinned at 253 for SINet and 127 for SINet-v2, so appended images displaced base-pool exposure; B and C10 had identical pool sizes and step counts.Source: B Detailed Method Specification · p. 16
  • The falsification shuffle was selected from predeclared permutations without inspecting outcomes; its rank correlation with the true score was −0.03351 versus CINV’s −1.0.A preregistered distinctness gate required a Jaccard bound of 0.50; paired allocations differed in 452–511 of 1000 injected images.Source: B Detailed Method Specification · pp. 16–17
  • The allocation sweep tested budgets N ∈ {250, 500, 1000, 2000, 3000, 4447} and extended τ until concentrated and uniform extremes were reached; the campaign used τ = 1.0 and N = 1000.Cluster scores are means of Equation 1 over target members; dividing by the score standard deviation makes τ scale-free, and largest remainders assign leftover integer quotas.Source: B Detailed Method Specification · p. 17
  • Sα at α = 0.5 is the primary endpoint, MAE on CAMO-250 selects checkpoints, and Fwβ and Eϕ are reported but do not decide outcomes.Campaign uncertainty uses pooled within-arm seed variability, with degrees of freedom determined by arms and seeds; bars are not exchanged across campaigns.Source: B Detailed Method Specification · p. 17
  • Campaign rules distinguish the committed four-arm, seed-expansion, oracle, optimization-schedule, positive-control, and falsification comparisons, with outcomes governed by campaign-specific verdict rules.Decision rules were committed before the runs they govern; three append-only amendments corrected reference-precision defects or marked an inapplicable verification gate without changing thresholds.Source: B Detailed Method Specification · pp. 17–18
  • A2’s paired photographs and masks differ in mean foreground fraction by 0.0058 across 1000 of 5447 images, affecting B − A2 but not the decisive C10 − B gap.B and C10 share mask provenance, pool size, and render pool.Source: B Detailed Method Specification · p. 17

C Decisive Gap in Every Cell, and the Bars It Is Read Against

Across the decisive comparisons, gaps remain within campaign-specific noise bars. The bars differ in how they are estimated, and expanded seed testing provides a sensitivity check.

C Decisive Gap in Every Cell, and the Bars It Is Read Against

  • Sixteen of sixteen decisive comparisons fall within noise, and none reaches half its campaign-specific bar.Table 5 reports the decisive gap across four architecture-endpoint cells, with each campaign read against its own pre-registered bar.Source: Table 5 · p. 18, C Decisive Gap in Every Cell, and the… · p. 18
  • The committed four-arm bar is 0.0179; A0 contributes 92.7% of pooled variance despite not appearing in the decisive comparison.A0's per-arm standard deviation is 0.01727, versus 0.00193 for A2, 0.00181 for B, and 0.00406 for C10.Source: C Decisive Gap in Every Cell, and the… · pp. 18–19
  • The falsification campaign's bar is 0.0055, while the seed expansion measures 0.0069 over the same four arms at 28 rather than 8 degrees of freedom.The paper reads the gap against the tightest defensible bar and uses the seed-expansion bar as its sensitivity statement.Source: C Decisive Gap in Every Cell, and the… · p. 19
  • The projected 8/3 bar reduction was erroneous: it applies to a standard error of a difference, not the verdict rule, and the eight-seed bar is more trustworthy.The pooled per-run standard deviation is not expected to depend on seed count; measured per-arm spreads grew, and the three-seed bars were optimistic rather than conservative.Source: C Decisive Gap in Every Cell, and the… · p. 19
  • The three-seed Welch interval for the decisive gap is [−0.0035, +0.0136] and excludes the reference effect, but its exclusion margin is thin.The eight-seed interval supersedes this estimate and is recorded for continuity with the committed account.Source: C Decisive Gap in Every Cell, and the… · p. 19
  • Across thresholds 0.4/0.5/0.6, concentration contrasts resolve at every threshold, while direction contrasts never exceed 0.52× and 0.14× their bars.Both concentration contrasts are sign-consistent at all three thresholds; neither direction contrast resolves anywhere in the sweep.Source: C Decisive Gap in Every Cell, and the… · p. 19
  • At eight seeds, five of sixteen comparisons reach equivalence at δ = 0.005, but the primary-cell targeting-versus-random gap does not.Four direction contrasts and the targeting-versus-random gap on SINet/NC4K reach equivalence; none did at three seeds.Source: C Decisive Gap in Every Cell, and the… · p. 20
  • The decisive comparison shares the entire base pool and roughly half of its injected images by construction.This shared material is identified as a property of comparing the two arms.Source: C Decisive Gap in Every Cell, and the… · p. 20

D Supplementary Tables

Supplementary analyses report clustering, signal-generality, area-control, run-level, and allocation diagnostics. The results include weak cluster separation, different signal orderings across aggregations, and disproportionate allocation to CAMO images.

D Supplementary Tables

  • No embedding-space k reaches a conventionally well-separated silhouette, and clipL224's selection criterion returns the grid edge and fails its declared guard.Table 7 reports silhouette, bootstrap ARI, and seed ARI across the declared k grid.Source: Table 7 · p. 21
  • The whole-image signal ordering passes 8/8 rows, while the boundary block passes 2/8 and has four negative rows.The boundary outcome falls outside the pre-registered interpretation space.Source: Table 8 · p. 21
  • After partialling object area, the ordering survives 7/8 and 6/8 rows against a 6/8 floor; ρ(1 −Sα) rises in every row.That increase is opposite to the object-area confound's prediction.Source: Table 9 · p. 22
  • Tables 10 and 11 report full-precision metrics for 39 of the 103 runs on COD10K-test and NC4K; re-scored B and C10 values reproduce exactly at six decimals.The same runs are shown for both endpoints.Source: Table 10 · p. 23, Table 11 · p. 24
  • CAMO is 24.8% of the target pool but receives 51.5% of the budget, a 2.08× enrichment under Table 12's allocation.The two largest quotas together take 33.5% of the budget, are 90.2% CAMO, and contain 7 of the 2026 primary-endpoint images.Source: Table 12 · p. 25

E Coverage of the Target Manifold, and Why AUC Is Not Used

The paper measures target-manifold coverage with k-NN recall and does not use a real-versus-synthetic classifier as evidence because re-encoding alone can separate identical images.

E Coverage of the Target Manifold, and Why AUC Is Not Used

  • Synthetic-pool k-NN recall is 0.4829/0.4834/0.1277 for the authors' pool and 0.4668/0.5421/0.1349 for the paper's renders across dinoL224/dinoL518/clipL224.The target comprises 4040 images; recall is the fraction inside the compared set's k-NN balls at k = 5.Source: E Coverage of the Target Manifold, and… · p. 26
  • Real sets cover the target near their own ceiling: target against random halves scores 0.9332/0.9431/0.8931, while COD10K/CAMO target parts score 0.8898/0.9003/0.8730.Values are listed in dinoL224/dinoL518/clipL224 order.Source: E Coverage of the Target Manifold, and… · p. 26
  • Re-saving identical target images at JPEG quality 30 yields AUC 0.9928 in clipL224, so the classifier is not used as evidence of a distributional gap.The two synthetic pools score 0.9993 and 0.9995, respectively.Source: E Coverage of the Target Manifold, and… · p. 26

F CHAMELEON Contamination: Calibration, Controls and Adjudication

The contamination analysis calibrates two matching tiers, checks their controls, and adjudicates geometric matches rather than treating keypoint agreement alone as proof of duplicate photographs. Its reported counts remain lower bounds.

F CHAMELEON Contamination: Calibration, Controls and Adjudication

  • Tier A groups images by pixel dimensions, shortlists 32 × 32 greyscale descriptors at RMS distance 14.0, then confirms full-resolution RGB mean absolute difference below 6/255.The descriptor is not contrast-normalized; the detector's confirmed-pair count saturates at 41 as tolerance increases.Source: F CHAMELEON Contamination:… · p. 26
  • Tier B fixes thresholds using 41 Tier-A positives and 4000 random CHAMELEON×training negatives before scoring unchecked images.It uses SIFT correspondences and a RANSAC homography at 4 px, with Lowe ratio 0.75.Source: F CHAMELEON Contamination:… · p. 26
  • At the 20-inlier operating point, Tier B recalls 41/41 known pairs and flags 0 of 120 NC4K controls.The operating point is one inlier above the NC4K negative maximum of 19; no unchecked image passes the resolution-normalized global-matching control.Source: F CHAMELEON Contamination:… · p. 26
  • Keypoint agreement alone cannot establish the same photograph, so candidate homographies are checked by warping images and measuring overlap residuals.On eight known pairs, the instrument reproduces recorded mean absolute differences to within 0.002 at H = I.Source: F CHAMELEON Contamination:… · p. 27
  • Animal-7 is excluded: its homography collapses the frame toward a line, despite clearing the operating point by five inliers.The NC4K negatives reached 19 inliers; the same-dimension Tier-A band is not an acceptance rule for resized or cropped copies.Source: F CHAMELEON Contamination:… · p. 27
  • The reported exact-hash zero applies only to the training pool; three byte-identical images match CAMO-250, the checkpoint-selection split.Those CAMO-250 files are absent from the training pool and therefore excluded from the contamination count.Source: F CHAMELEON Contamination:… · p. 27
  • Sixteen images have no partner at either tier and are reported unchecked, not clean; different photographs of one specimen, montages, and heavy color edits are not detected.The reported figures are lower bounds.Source: F CHAMELEON Contamination:… · p. 27

G CHAMELEON Mask Releases and Detector Validation

The two CHAMELEON ground-truth releases differ substantially, and the release chosen changes the measured MAE on identical predictions. The released detector also passes a synthetic known-answer test and reproduces the canonical duplicate pairs and names.

G CHAMELEON Mask Releases and Detector Validation

H Three Controls: Execution Records and Disclosures

Three control campaigns disclose important design choices and execution details. They include a test-label oracle, a bundled optimization-schedule change, and a separately preregistered seed expansion.

H Three Controls: Execution Records and Disclosures

  • CORACLE reassigns the committed allocation across clusters by their true endpoint error, using test labels as a diagnostic upper bound rather than a real method.The oracle uses one architecture’s errors for both; the architectures’ per-cluster errors have rank correlation 0.9211, making the second architecture’s oracle near-perfect rather than perfect.Source: H Three Controls: Execution Records… · p. 28
  • Unpinning the optimization budget bundles four changes, and roughly one-third more budget made every arm slightly worse, with 11 of 12 cross-schedule comparisons negative.Steps per epoch increased from 253 to 341 for SINet and 127 to 171 for SINet-v2; learning-rate decay remained per epoch and unrescaled.Source: H Three Controls: Execution Records… · p. 28
  • The seed expansion was a separate campaign with four arms, its own noise estimate and verdicts, and eight seeds fixed in writing before its first run.The original 24 runs were rescored, not retrained, and had to reproduce their committed values to six decimal places.Source: H Three Controls: Execution Records… · p. 28
  • Two runs interrupted by host power loss were discarded and restarted from scratch; no partially trained model entered scoring.The campaign cost 75.9 GPU-hours across 40 trained runs.Source: H Three Controls: Execution Records… · p. 28
  • Across seed-expansion runs, pseudo-labeling appended different numbers of differently labeled images per arm, while step counts stayed pinned.Mean appended-image counts were 1977 (B), 2025 (C10), 1990 (CSHUF), and 1979 (CINV); per-run values ranged from 1817 to 2259.Source: H Three Controls: Execution Records… · pp. 28–29
Loading 2610.09807v1…