A Systematic Evaluation of the COTQ Provincial Land Cover Product: Structural Consistency, Spectral Separability, and Relative Positioning Against ESA, ESRI, and Google Products

Key Policy Highlights

• COTQ provides a more detailed representation of artificial land transformation and appears well suited to Québec’s operational monitoring needs.The study argues that a regional product can address ecological and operational requirements not fully captured by globally standardized datasets.Source: Key Policy Highlights · p. 2
• ESA WorldCover is the closest global analogue to COTQ, but it underestimates heterogeneous urban and industrial surfaces.The evaluation framework is intended to provide transferable regional validation when authoritative ground truth is absent.Source: Key Policy Highlights · p. 2

1. Introduction

• Global products show discrepancies in boreal, subarctic, and temperate regions, including confusion among rocky, sparse-vegetation, and cryptogamic surfaces, wetland ecotones, and small anthropogenic features.Their worldwide uniformity does not reflect locally defined class schemes or operational priorities.Source: §1 · p. 3
• Québec requires an annual province-wide 10-m map for land occupation and soil artificialisation, but global products can miss small, fragmented urban and infrastructure patterns.Provincial control of production supports flexible class definitions, reproducibility, and stable annual updates.Source: §1 · p. 3
• COTQ is a dedicated 10-m Québec product produced from Sentinel-2 imagery using deep learning-based semantic segmentation.Its production methodology is documented elsewhere; this paper focuses on evaluating the resulting product.Source: §1 · pp. 3–4
• The study compares COTQ with ESA WorldCover, Google Dynamic World, and ESRI Land Cover after harmonizing their classes under a common legend.The global products are established external benchmarks, but their published accuracies cannot be transferred directly to Québec because validation designs, references, and class definitions differ.Source: §1 · pp. 3–4
• The evaluation combines structural, spectral, and temporal-consistency metrics with product comparison and expert interpretation of sampled disagreement areas.This multi-angle approach addresses the difficulty of relying on a single accuracy indicator when independent, representative, interpretable reference data are hard to obtain.Source: §1 · p. 4

2.1 The COTQ taxonomy

• The COTQ taxonomy balances physical land-cover realism, reliable 10-m discriminability, and Québec-specific monitoring priorities, including explicit urban categories for roads, residential areas, and industrial or commercial transformations.It separates bare earth from exposed rock, merges grasslands and shrublands into Low Vegetation, omits an explicit Crops class, and adds Moss & Lichen and Wetlands; Ice/Snow is not retained in the final annual product.Source: §2.1 · p. 5

2.2 Data preparation and model training

• The COTQ extends a semi-supervised annotation and training pipeline introduced for southern Québec by (Clabaut et al., 2024) to province-wide operational production.The overview presents the pipeline as context for interpreting COTQ’s behaviour relative to WorldCover, ESRI, and Google products.Source: §2.2 · pp. 6–7

2.2.1 Building the synthetic dataset

• DeepResUNet performed best among the three tested architectures, while all three benefited from synthetic training data and showed strong mapping capability before fine-tuning on real imagery.The pipeline reduced annotation effort by selecting homogeneous superpixels and assembling them into spectrally faithful but spatially varied mosaics. Cloud and shadow errors could still bias annual composites, so NDVI and NDWI were added as illumination-invariant cues to improve temporal stability and class consistency.Source: §2.2.1 · p. 8

2.2.2 Data augmentation

• Dedicated augmentation using real cloud rasters nearly eliminated systematic cloud- and shadow-related artefacts, stabilizing per-date predictions for temporal compositing.Semi-transparent clouds were simulated with a spatially varying transparency field, while the corresponding masks were updated to label cloud presence.Source: §2.2.2 · pp. 9–10, Figure 2 · p. 9

2.3 Temporal aggregation and majority-vote compositing

• Pixel-wise majority voting combines valid per-date predictions into annual maps, reducing isolated classification artefacts while preserving genuine land-cover changes and traceability to Sentinel-2 observations.NoData, Cloud, and Ice/Snow predictions are excluded; previous-year information is limited to Forest and Low Vegetation, with extra rules for near-balanced ambiguous cases.Source: §2.3 · p. 11, Figure 4 · p. 11

2.4 Active learning process

• Four active-learning iterations used representative corrected areas to address recurring errors, producing clear visual gains in separating anthropogenic surfaces from rocky and cryptogamic substrates despite the absence of suitable reference data for quantification.The example error pattern was dramatically corrected after the first iteration.Source: §2.4 · p. 12
• An in-house editing tool lets operators correct 512×512-pixel patches using polygon replacement, spectral thresholds, or freehand lines, then feeds georeferenced edits into synthetic mosaics.The raster editor supports undo through a history stack; each corrected patch produces image and updated-mask GeoTIFFs.Source: §2.4 · p. 12
• Ice/Snow and Moss & Lichen were added to represent northern environments; high-latitude samples were automatically mosaicked into training images without re-annotating large datasets.The semi-supervised sample-collection and synthetic-mosaicking pipeline was developed in (Clabaut et al., 2024).Source: §2.4 · pp. 12–13
• Active learning and a dedicated moss-lichen class strongly reduced confusion between residential surfaces and cryptogam-mixed low vegetation, improving ecological detail.Figure 5 contrasts the Version 1 classification with the result after active-learning corrections and the addition of the moss-lichen class.Source: Figure 5 · p. 13, §2.4 · p. 14
• The annual production chain spans sample selection, synthetic-mosaic generation, cloud–shadow augmentation, DeepResUNet training, active learning, per-date inference, and temporal majority-vote compositing.The workflow produces the final annual 10-m COTQ land-cover product for Québec.Source: §2.4 · p. 14, Figure 6 · p. 14

3.1 Evaluation Framework in the Absence of Ground Truth

• Because no reliable large-scale ground truth is available, the study uses global products as comparative benchmarks and targeted photo-interpretation to assess key disagreements.The global products are not treated as authoritative representations of Québec.Source: §3.1 · p. 14

3.2 Choice of Sentinel-2 Evaluation Areas

• Eight Sentinel-2 footprints were selected to span Québec’s major bioclimatic domains and land-cover diversity, from southern landscapes to the northern shield.The sites were chosen to maximize ecological diversity, thematic representation, and landscape heterogeneity within a limited number of large footprints.Source: §3.2 · pp. 15–16
• Sites A and B represent complex southern landscapes; B also covers Lac Saint-Pierre wetlands, where an earlier COTQ version classified the area entirely as forest.The sites address urban and agricultural mosaics, fragmented landscapes, wetland detection, shoreline marshes, flooded meadows, and dynamic hydrological regimes.Source: §3.2 · pp. 15–16
• Sites C–H extend coverage through mixed and boreal forests to forest-tundra and Low Arctic Tundra, including wetlands, disturbances, rocky surfaces, mosses, and lichens.These sites test behaviour across ecological gradients and contrasting northern environments.Source: Figure 7 · p. 15, §3.2 · p. 16

3.3 Semantic harmonization of Land Cover Classes

• All products are remapped to seven COTQ classes: Forest, Low vegetation, Rock/Bare, Built-up, Water, Wetland, and Ice/Snow.The raster class codes are 3, 4, 6, 7, 9, 10, and 11, respectively.Source: §3.3 · p. 16
• The shared legend supports comparison within a common operational framework aligned with COTQ’s objectives, rather than evaluation of the original thematic legends.Some distinctions in the source products are intentionally simplified, so results concern end-use suitability under the shared legend.Source: §3.3 · p. 16
• Table 2 presents the mapping rules used to harmonize each product’s classes to the COTQ legend.Its caption identifies the table’s scope as cross-dataset land-cover harmonization.Source: §3.3 · p. 16, Table 2 · p. 17

3.4 Inter-Product Structural Comparison

• The structural analysis characterizes internal consistency, closeness to COTQ segmentation, object size, spatial detail, and boundary complexity—not accuracy.It uses object-size distributions, shape complexity, and Adjusted Rand Index.Source: §3.4 · p. 17

3.4.1 Object-Size Distribution (OSD)

• Object-size distributions summarize connected-component sizes by class; many small objects may indicate fine detail or noise, while large objects may indicate under-segmentation.Comparing distributions reveals systematic structural differences independently of spectral content.Source: §3.4.1 · p. 18

3.4.2 Shape Complexity (Compactness Measure)

• Comp(S) = P(S)^2/(4πA(S)) measures object shape complexity; higher values indicate more complex boundaries, complementing object-size distributions.A(S) is object area and P(S) its perimeter; irregular shapes can reflect fine spatial structures.Source: §3.4.2 · p. 18

3.4.3 Adjusted Rand Index

• ARI compares two image partitions by whether pixel pairs are grouped together in both maps, with a correction for chance agreement.It ranges from −1 to 1: 1 indicates identical object partitioning, values near 0 indicate chance-level agreement, and negative values indicate worse-than-random correspondence.Source: §3.4.3 · pp. 18–19

3.4.4 Intersection over Union

• Class-specific IoU measures the shared mapped area relative to the area assigned that class by either product, penalizing both overestimation and underestimation.A score of 1 indicates complete spatial overlap; 0 indicates none.Source: §3.4.4 · p. 19

3.5 Spectral Separability Metrics

• Spectral samples use the same summer 2021 Sentinel-2 image per area, with up to 20,000 valid pixels per product and equal weighting of the eight areas.No class balancing is applied; the metrics describe how products partition regional spectral spaces rather than estimate province-wide properties. Silhouette and Calinski–Harabasz use area-level standardized bands, while SAM uses L2-normalized spectra.Source: §3.5 · pp. 19–20

3.5.1 Silhouette Coefficient

• Silhouette scores compare each pixel’s within-class compactness with its separation from the nearest alternative class, capturing local consistency and boundary ambiguity.Scores near 1 indicate strong compactness and separation; values near zero indicate overlap, while negative values may signal a potential misclassification or inadequate class structure.Source: §3.5.1 · p. 20

3.5.2 Calinski-Harabasz Index

• The Calinski–Harabasz index compares between-class dispersion with within-class dispersion, summarizing global spectral separability rather than sample-wise consistency.Higher values indicate sharper class boundaries and greater compactness; the index complements Silhouette by assessing global variance structure.Source: §3.5.2 · p. 21

3.5.3 Spectral Angle Mapper (SAM)

• Spectral Angle Mapper measures angular similarity between spectral vectors, emphasizing reflectance-shape differences rather than overall brightness.Its brightness and illumination insensitivity makes it suitable for landscapes with strong illumination variation, including rugged or northern environments and bright mineral surfaces.Source: §3.5.3 · pp. 21–22

3.6.1 Discrepancy map

• A three-pixel erosion removes narrow disagreement ridges, while urban disagreements are evaluated before erosion because the fragmented class contains meaningful fine-scale patterns.The urban assessment also compares resampled products with a 50-cm classification derived from 2022 Jilin-1 imagery, used as a proxy reference after visual inspection found no significant development or demolition between 2021 and 2022.Source: §3.6.1 · p. 22

3.6.2 Random Sampling of Disagreements

• Forty points per Sentinel-2 footprint are randomly sampled from eroded disagreement regions and interpreted from Google Earth VHR imagery before product labels are revealed.One analyst with more than 10 years of remote-sensing experience assigns a reference class; indeterminate points are excluded from the class-based comparison, which summarizes relative support for COTQ and WorldCover.Source: §3.6.2 · pp. 22–23
• The evaluation framework compares four land-cover products using object-based and spectral metrics, followed by focused COTQ–WorldCover disagreement analysis and photo-interpretation.Figure 8 summarizes this sequence and its overall quality-assessment stage.Source: Figure 8 · p. 23

4.1.1 Object-Size Distribution (OSD)

• Google and ESRI had fewer small objects than WorldCover and COTQ, whose log-log trends were stronger (R² 0.79–0.98).The log-log pattern is a comparative structural indicator, not formal evidence of a power-law distribution; visual inspection also found small lakes, narrow roads, and isolated buildings often absent from ESRI and Google.Source: §4.1.1 · pp. 23–24, Figure 9 · p. 24

4.1.2 Shape Complexity

• WorldCover and COTQ produced finer, more fragmented geometries; built-up boundary complexity was approximately five times higher than in ESRI and Google.The authors suggest post-classification generalization may explain the smaller-object scarcity and simpler boundaries in ESRI and Google.Source: §4.1.2 · p. 25, Figure 10 · p. 25

4.1.3 Built-Up class: localized reference-based assessment

• 67.5% IoU for COTQ versus 52.5% for WorldCover against the 50-cm reference; COTQ had the highest agreement among the four products.In the localized built-up comparison, visual assessment also suggested COTQ better matched built-up structure, while WorldCover underestimated urban surfaces in vegetated neighborhoods.Source: §4.1.3 · pp. 26–27, Table 3 · p. 27

4.1.4 Adjusted Random Index and Intersection over Union

• COTQ–WorldCover agreement was highest: ARI 0.75 versus 0.67–0.68 for other pairs, and mIoU 82% versus 74%–76% for COTQ with ESRI or Google.Source: §4.1.4 · p. 28, Figure 13 · p. 28, Figure 14 · p. 29

4.1.5 Selection of a reference product selection

• Structural indicators identified ESA WorldCover as the global product closest to COTQ, while ESRI and Google showed stronger spatial generalization.Across the indicators, COTQ and WorldCover preserved more small objects and more complex boundaries.Source: §4.1.5 · p. 29

4.2.1 The Silhouette coefficient

• Water had Silhouette coefficients around 0.45–0.50, while Low Vegetation, Bare Soil/Rock, Built-Up, and Wetland were negative; SAM angles were lowest for Water, Bare Soil/Rock, and High Vegetation.Built-Up had the highest SAM variability across products, at 0.105–0.145 radians.Source: §4.2.1 · p. 31, §4.2.3 · p. 32

4.2.5 Conclusion of the Spectral Separability Analysis

• Across three spectral indicators, COTQ and ESA WorldCover showed the strongest separability, with more compact clusters than ESRI Land Cover and Google Dynamic World.The results characterize spectral organization, not direct thematic correctness, and cover only four-band Sentinel-2 reflectance space.Source: §4.2 · p. 30, §4.2.5 · p. 34

4.3.2 Photo-interpretation outcomes within disagreement areas

• Among 320 photo-interpreted disagreement points, COTQ was more often supported for Low Vegetation, Rock/Bare Soil, Built-Up, Water, and Wetland; WorldCover was more often supported for Forest.These outcomes describe relative support within sampled disagreement areas, not class-specific accuracy across the full study domain.Source: §4.3.2 · p. 35, Table 5 · p. 35
• The class-based photo-interpretation comparison counts only observations where imagery confidently supports either COTQ or WorldCover, excluding indeterminate cases and cases where neither product is judged correct.Figure 18 presents outcomes within sampled substantive disagreement areas; its caption supplies no class-level counts.Source: Figure 18 · p. 36

4.3.3.1 Urban and anthropogenic surfaces

• COTQ correctly labels an asphalt parking lot Built-Up where WorldCover maps Low Vegetation, but quarry and earthwork labels differ defensibly by physical-cover versus artificialization definitions.COTQ prioritizes identifying human-driven land-use change as Built-Up, while WorldCover classifies exposed soil or rock by its physical substrate.Source: §4.3.3.1 · pp. 36–37, Figure 19 · p. 37

4.3.3.2 Wetlands: peatlands vs. riparian wetlands

• WorldCover better preserves shoreline wetlands, while COTQ more accurately delineates coherent peatland and wet-meadow complexes; neither product is uniformly superior across wetland subclasses.COTQ maps parts of a fine-scale shoreline wetland as Forest, whereas WorldCover fragments a peatland and labels substantial areas Low Vegetation.Source: §4.3.3.2 · pp. 37–38, Figure 20 · p. 38

4.3.3.3 Moss, lichen, and low vegetation over bedrock

• In northern rocky environments, WorldCover assigns moss- and lichen-covered surfaces to Low Vegetation, while COTQ predominantly labels them Rock/Bare, which the reference evidence supports at 10-m scale.Because the other products lack a Moss & Lichen class, these surfaces were grouped under Low Vegetation for comparison; this interpretation does not establish that cryptogamic vegetation is biologically absent. Uncertain cases were excluded as indeterminate.Source: §4.3.3.3 · pp. 38–39, Figure 21 · p. 39

5 Conclusion

• Structurally, COTQ and ESA WorldCover preserve small objects and complex boundaries, with finer segmentation consistency than ESRI Land Cover and Google Dynamic World.These properties are relevant to detecting subtle anthropogenic changes, narrow wetlands, and sparse northern vegetation.Source: §5 · pp. 40–41
• Spectral metrics likewise place COTQ and WorldCover among the products with the most compact, best-separated Sentinel-2 clusters, but the four-band analysis is not direct evidence of accuracy.It omits spatial context and temporal information and is presented as a complementary diagnostic.Source: §5 · p. 41
• Photo-interpretation found that WorldCover can better preserve narrow gradients in some riparian wetlands, while COTQ more faithfully represents intermixed built-up areas.Differences also reflect thematic definitions and priorities; northern disagreements chiefly concern rocky surfaces partly covered by cryptogamic vegetation.Source: §5 · p. 41
• The evaluation is constrained by the lack of province-scale ground truth and by high-resolution imagery limitations, particularly in northern cryptogamic environments.Some observed differences represent alternative internally consistent interpretations rather than unequivocal classification errors.Source: §5 · pp. 41–42
• The combined evaluation framework may transfer to other jurisdictions facing similar reference-data, class-definition, and interoperability challenges, although parts of this study reflect Québec-specific conditions and priorities.The authors summarize the relative assessment of all four products in Table 6.Source: §5 · pp. 42–43, Table 6 · p. 42
• Within the evaluated Québec conditions and monitoring objectives, COTQ offers the strongest overall combination of spatial detail, thematic suitability, comparative support, and annual updates among the four products.The authors identify it as the preferred operational product for monitoring land occupation and soil artificialisation.Source: §5 · pp. 41, 43

Data Availability

• The COTQ map and reference products are publicly available for visual inspection through the listed Earth Engine application.The passage provides the application URL.Source: Data Availability · p. 45
• Sentinel-2 imagery is distributed by the European Space Agency.Source: Data Availability · p. 45
• Derived evaluation metrics and scripts are available from the authors upon reasonable request.Source: Data Availability · p. 45

Declaration of Interest

• The authors declare no known competing financial or non-financial interests that could have influenced the reported work.Source: Declaration of Interest · p. 45
• The ministry that supported COTQ development was not involved in study design, analysis, interpretation, or manuscript writing.Source: Declaration of Interest · p. 45

Use of Generative Artificial Intelligence

• AI language models assisted with translating the manuscript from French into English.Source: Use of Generative Artificial… · p. 45
• Generative AI was also used partially to support development of certain analysis-code components.Source: Use of Generative Artificial… · p. 45
• The authors state that they reviewed, verified, and edited AI-assisted outputs and take responsibility for the manuscript and associated code.Source: Use of Generative Artificial… · p. 45

• E.C., S.F., and Y.B. are credited with conceptualization.Source: Author contributions statement · p. 45
• E.C. is credited with writing the original draft.Source: Author contributions statement · p. 46
• E.C., S.F., and Y.B. are credited with review and editing.Source: Author contributions statement · p. 46