Source-linked AI summary
Segmentation of Bovid Dentition Under Imperfect Annotations: A Comparative Study of Convolutional and Attention Models
Keith G. Mills, Evan B. Sanders, Gregory J. Matthews, Juliet K. Brophy
TL;DR
The paper addresses semantic segmentation when bovid-tooth masks are handcrafted, resolution-mismatched, or spatially misaligned. It compares convolutional and transformer architectures on B.O.V.I.D. while testing preprocessing and alignment strategies, finding limited quantitative sensitivity but substantial qualitative differences in predicted masks.
Problem
B.O.V.I.D.’s handmade masks were not created for machine-learning training, introducing resolution mismatches and spatial misalignment that complicate supervised segmentation.
Method
The study compares convolutional and transformer-based segmentation architectures and evaluates pruning, centroid alignment, contrast processing, and downsampling on B.O.V.I.D.
Results
Preprocessing choices have limited effects on Dice and mIoU but substantial qualitative effects, while lightweight CNN encoders achieve broadly comparable quality to larger transformer models.
Takeaways & Limitations
B.O.V.I.D. can support supervised segmentation after lightweight preprocessing, and a lightweight CNN can provide segmentation quality broadly comparable to a larger transformer.
Takeaways & Limitations
Correcting image-mask cropping or misalignment requires identifying tooth-mask overlap, which is the segmentation objective itself.
Abstract
from arXiv · showhide
Semantic segmentation decomposes an image into distinct mask regions corresponding to different object categories, such as people, cars, signs or buildings. Advances in machine learning (ML) have shifted this task away from traditional rule-based heuristics such as edge detection, towards deep neural networks (DNN) that learn to classify pixels directly. However, semantic segmentation DNNs crucially depend on expertly designed mask targets to learn from, and imperfect or misaligned masks can interfere with a model's ability to learn effectively. This paper presents a comparative study of segmentation architectures, ranging from convolutional backbones to vision transformers, applied to the B.O.V.I.D. dataset, a corpus of high-resolution bovid dental photographs paired with hand-made segmentation masks not originally designed for ML-based training. We evaluate a range of preprocessing and alignment techniques to mitigate the resulting label imperfections. We find that while these preprocessing choices have limited effect on quantitative metrics such as Dice score and mIoU, their qualitative impact on predicted masks is substantial.
1. Introduction
The paper frames bovid-tooth segmentation as a machine-learning problem complicated by imperfect dataset masks. It compares segmentation architectures and preprocessing techniques, finding that qualitative mask differences can be substantial even when quantitative metrics change little.
- Bovid teeth support paleoenvironmental reconstruction and taxonomic identification because Bovidae have relatively strict ecological tendencies and abundant fossil teeth.
- Morphometric analyses of tooth form, size, and occlusal-surface shape differentiate bovid taxonomic tribes and species.
- The paper investigates machine-learning methods for isolating bovid teeth from raw color photographs and producing binarized images for analysis.
- The study compares semantic-segmentation encoders spanning ResNets, MobileNets, EfficientNets, and Vision Transformer-based backbones across size and complexity.
- Experiments use B.O.V.I.D.’s handmade masks and evaluate preprocessing, alignment, out-of-distribution data, and taxonomic variation to address imperfect training targets.
- Preprocessing choices have limited effects on Dice score and mIoU but substantial qualitative effects, while MobileNetV2 is comparable to larger SegFormer encoders.
2. Bovidae Extant Dataset
B.O.V.I.D. pairs high-resolution bovid-tooth photographs with handcrafted binary masks and taxonomic metadata. Because the dataset was not designed for machine-learning segmentation, resolution mismatches and spatially misaligned masks complicate supervised training.
- B.O.V.I.D. contains 3592 extant Bovidae tooth images spanning seven tribes, with substantial variation in image resolution.
- Each tooth has a raw color image of its occlusal surface and a handcrafted black-and-white mask, with associated taxonomic metadata.
- Raw images are model inputs, mask images are supervised labels, and pretrained off-the-shelf segmentation models are fine-tuned for the task.
- Unlike classical segmentation datasets, B.O.V.I.D. contains raw-mask pairs with resolutions that may differ or have incompatible aspect ratios.
- Some equal-resolution pairs still place the mask away from the tooth’s location in the raw image, heavily disrupting supervised learning.
- Correcting crops or spatial misalignment requires identifying tooth-mask overlap, which is itself the segmentation objective being learned.
3. Augmentations
The paper prepares B.O.V.I.D. for segmentation by pruning incompatible pairs, centering raw images to their masks, and applying contrast and resolution transformations. Training uses binary cross-entropy combined with weighted Dice loss.
- 3.1. Dataset Pruning & Cropping: Pairs with unequal or non-proportional resolutions are removed, eliminating 283 pairs and leaving 3325 image pairs.
- 3.1. Dataset Pruning & Cropping: Foreground-centering aligns raw images to binary masks by thresholding foreground pixels, estimating their centroid, and extracting a centered crop.
- 3.1. Dataset Pruning & Cropping: The crop is resized to the mask’s exact dimensions, producing pixel-compatible image-mask pairs for segmentation.
- 3.1. Dataset Pruning & Cropping: The resulting 3325-pair dataset becomes the basis for experiments after pruning and augmentation.
- 3.2. Online Preprocessing and Training: Masks are loaded in grayscale, inverted so the target region is white, and represented as one foreground channel with values in {0, 1}.
- 3.2. Online Preprocessing and Training: Contrast processing uses histogram equalization, CLAHE, or no equalization; images and masks may also be downsampled by integer factor d.
- 3.2. Online Preprocessing and Training: Models are trained with a combination of binary cross-entropy and weighted Dice losses, controlled by configurable scalar λ_Dice.
4. Results and Discussion
Experiments compare SegFormer and U-Net++ with multiple encoders, preprocessing methods, per-tribe evaluation, and out-of-distribution images. Quantitative metrics often appear similar across preprocessing choices, while qualitative mask quality reveals meaningful differences between techniques and architectures.
- Experimental setup: The study evaluates an 80%/20% split using SegFormer MiT-B3 and U-Net++ with MobileNetV2, ResNet-34, or EfficientNet-B0 encoders.The experiments use semantic segmentation models from segmentation-models-pytorch and include supplementary hyperparameter ablations.
- SegFormer results: SegFormer preprocessing variants show similar Dice trends, but CLAHE c = 25 consistently achieves lower test BCE loss than the alternatives.The compared methods are CLAHE c ∈[10, 25], histogram equalization, and no equalization over 70 training epochs.
- SegFormer results: SegFormer heatmaps generally center on the tooth region, while preprocessing changes affect shape fidelity, with CLAHE 10 best capturing one complex shape and HE or NE missing details in another.The ground-truth mask location can be shifted relative to the raw image, making qualitative conformity important beyond metric similarity.
- MBv2 results: MBv2 training is initially more stochastic than SegFormer, with larger metric spikes that smooth out by approximately epoch 50.Its BCE loss, Dice loss, and Dice metric follow broadly similar directional trends, but its best metric bounds are lower than SegFormer’s.
- MBv2 results: For MBv2, CLAHE c = 10 best represents complex tooth shapes, whereas NE performs well on simpler images but fails on complex examples.Figure 6 qualitatively supports the importance of preprocessing strategy for MBv2 mask accuracy.
- Stratified per-tribe results: Across seven tribes, CLAHE c = 25 achieves the best SegFormer Dice and mIoU in a majority of cases, although visual comparisons remain necessary.The strongest quantitative results occur for Alcelaphini, Antilopini, and Hippotragini, and are not correlated with dataset size.
5. Related Work
The study situates its comparison between established convolutional segmentation backbones and newer attention-based encoders, while focusing on how architectures tolerate B.O.V.I.D.'s non-ML-oriented annotations. It also evaluates contrast enhancement and lightweight alignment as preprocessing choices.
- Convolutional encoder-decoder networks remain common for pixel-level segmentation, using ResNet, MobileNet, and EfficientNet backbones especially for resource-constrained inference.
- Vision Transformers and transformer-based heads such as SegFormer provide an attention-based alternative whose efficacy this study evaluates through architecture choice.
- B.O.V.I.D. differs from conventional segmentation benchmarks because its expert-produced masks were created for morphometric analysis rather than ML training and consequently contain distinctive annotation issues.
- The preprocessing study uses contrast-limited adaptive histogram equalization and lightweight centroid alignment instead of a full deformable registration pipeline.
6. Conclusion
The paper concludes that lightweight preprocessing can make B.O.V.I.D. usable for supervised segmentation despite resolution mismatches and raw-mask misalignment. Quantitative metrics are largely insensitive to contrast-preprocessing choice, whereas predicted-mask quality differs substantially.
- Lightweight preprocessing makes B.O.V.I.D. viable for supervised segmentation despite resolution mismatches and raw-mask misalignment.
- Dice and mIoU are largely insensitive to the choice of contrast preprocessing, while qualitative differences in predicted masks are substantial.
- The conclusion frames the dataset's handcrafted masks as usable for segmentation after preprocessing addresses their mismatches and misalignment.
Supplementary Materials
The supplementary materials provide additional experimental details and analyses beyond the main manuscript.
- The supplementary materials enumerate the hyperparameters used in the experiments.
- They also provide additional ablations to complement the main experimental results.
- Additional visualizations are included alongside the supplementary experimental analyses.
A. Experimental Hyperparameters
The experiments use fixed training and optimization settings while varying batch size by model size and learning rate by model family. Evaluation uses an 80%/20% split, with controlled seeds for stratified analyses.
- Models are trained for 70 epochs with AdamW, weight decay 1e−5, and gradient-norm clipping on a single Nvidia DGX Spark unit.
- CNNs use a default learning rate of 1e−3, whereas transformers use 1e−4; batch size is 8 or 6 depending on model size.
- The dataset uses an 80%/20% training/testing split, with optional per-tribe stratification preserving tribe proportions.
- Stratification experiments use three controlled random seeds and report the resulting mean and standard deviation.
B. CNN Model Comparison
The study compares CNN backbones for segmentation and finds MobileNetV2 qualitatively more reliable than ResNet-34 and EfficientNet-B0 on representative heatmaps.
- CNN backbone comparison: MobileNetV2, ResNet-34, and EfficientNet-B0 are compared as U-Net++ encoder backbones without preprocessing.The comparison profiles older, larger, and more advanced CNN designs on the same task.
- Qualitative results: MobileNetV2 is the only backbone that consistently produces heatmaps vaguely resembling the ground-truth masks.ResNet-34 succeeds on one image but otherwise produces no mask or blotches.
- Qualitative results: EfficientNet-B0 fails to visualize the first mask at all.
C. Ablation Study on λDice
The ablation examines λDice and stratified MBv2 performance, showing a metric trade-off across loss terms and no single preprocessing setting dominating tribe-level results.
- λDice ablation: λDice = 0.5 yields lower Dice loss and higher Dice metric, but higher BCE loss than λDice = 0.3.
- Qualitative ablation: λDice = 1.0 produces excessively large masks with poorly defined borders, whereas λDice = 0.5 often produces inaccurate or absent heatmaps.Among the considered settings, λDice = 0.3 is identified as most optimal, with BCE potentially helping identify regions of interest.
- Stratified results: For MBv2, CLAHE c = 10 and NE each achieve the best performance on some tribes, while HE is never best.The tribe-level pattern is consistent between Dice and mIoU, and CLAHE c = 25 has no clear majority.
E. Additional OOD Examples
Additional examples present heatmaps across preprocessing settings, CNN comparisons, λDice values, and stratified metrics, including out-of-distribution samples from South Africa.
- OOD examples: SegFormer MiT-B3 produces competent out-of-distribution heatmaps with CLAHE clip limits c = 10 and c = 25.The c = 25 setting has a slight qualitative edge on several samples, including the first, second, and final images.
- Metrics and ablations: Figures 9, 11, and 12 visualize BCE loss, Dice loss, and Dice metrics across CNN backbones and λDice settings.The figures distinguish lower-is-better losses from the higher-is-better Dice metric; Table 2 reports per-tribe Dice and mIoU for MBv2.
- CNN examples: Figure 10 compares raw images, ground-truth masks, and inferred heatmaps from MBv2, ResNet-34, and EfficientNet-B0 without histogram equalization.Yellow denotes high tooth likelihood and purple denotes low likelihood.