Count-ception: Counting by Fully Convolutional Redundant Counting

Joseph Paul Cohen; Genevieve Boucher; Craig A. Glastonbury; Henry Z. Lo; Yoshua Bengio

Source-linked AI summary

Count-ception: Counting by Fully Convolutional Redundant Counting

Joseph Paul Cohen, Genevieve Boucher, Craig A. Glastonbury, Henry Z. Lo, Yoshua Bengio

arXiv:1703.08710v2Mar 25, 2017cs.CVcs.LGstat.ML

TL;DR

Counting in digital images is laborious and error-prone, motivating systems that return counts with localization. Count-ception replaces density-map prediction with redundant count maps processed fully convolutionally, and outperforms existing approaches while retaining performance on complicated cell structures.

Problem
Manual object counting is time-consuming and prone to fatigue-related errors, motivating automated counts with object-localization justification.
Method
Count-ception predicts redundant counts over receptive-field windows, averages overlapping predictions to recover the count, and replaces density maps with redundant counting.
Results
Count-ception outperforms existing approaches, and redundant counting improves accuracy compared with eliminating redundancy.
Takeaways & Limitations
The approach is promising for counting objects of different sizes in complicated structures, including adjoining cell walls.
Takeaways & Limitations
Redundant counting sacrifices exact x, y localization, and correct overall counts may arise without correctly identifying individual cells.

Abstract

from arXiv · show

Counting objects in digital images is a process that should be replaced by machines. This tedious task is time consuming and prone to errors due to fatigue of human annotators. The goal is to have a system that takes as input an image and returns a count of the objects inside and justification for the prediction in the form of object localization. We repose a problem, originally posed by Lempitsky and Zisserman, to instead predict a count map which contains redundant counts based on the receptive field of a smaller regression network. The regression network predicts a count of the objects that exist inside this frame. By processing the image in a fully convolutional way each pixel is going to be accounted for some number of times, the number of windows which include it, which is the size of each window, (i.e., 32x32 = 1024). To recover the true count we take the average over the redundant predictions. Our contribution is redundant counting instead of predicting a density map in order to average over errors. We also propose a novel deep neural network architecture adapted from the Inception family of networks called the Count-ception network. Together our approach results in a 20% relative improvement (2.9 to 2.3 MAE) over the state of the art method by Xie, Noble, and Zisserman in 2016.

1. Introduction

The paper targets machine-based object counting that reduces the time and fatigue burden of annotation while providing localization. It replaces complex segmentation pipelines and density-map prediction with redundant counting to average over errors.

Object counting is time-consuming and vulnerable to human annotator fatigue, motivating automated systems that also justify predictions through localization.
Classical counting segments objects from backgrounds, but overlapping objects require methods such as watershed transformation.These approaches also involve task-specific hyperparameters and are complicated to build.
Density-map methods assign each object density 1 so summing the map yields the total count while naturally handling overlap.
The paper proposes redundant counting instead of density maps to average over errors.

2. We propose a novel construction of networks and train-

The method predicts redundant counts within receptive fields using fully convolutional processing, replacing Gaussian density targets with count-map targets. It introduces Count-ception, an Inception-adapted regression architecture for this task.

Count-ception is a novel architecture adapted from the Inception family for the counting regression network.
The regression network predicts the number of objects inside each receptive field, producing a redundant count map through fully convolutional processing.Each pixel is counted as many times as the number of windows that include it, and averaging predictions recovers the true count.
32 × 32 = 1024 overlapping windows account for each pixel, providing the redundancy used to average predictions.
The approach uses deep convolutional networks and pads image borders so the receptive field can redundantly count objects.
Square-kernel count targets match the regression network’s receptive field, unlike Gaussian targets that require location-dependent values.

2. Related Work

The paper builds on density maps, fully convolutional counting, and receptive-field counting while distinguishing redundant count maps from related proximity and segmentation approaches.

Density-map counting began by predicting a map whose sum represents the object count, whereas this paper predicts redundant counts designed to tolerate prediction errors.
Density maps use Gaussian values centered on point annotations, requiring predictions to vary with distance and account for objects within the receptive field.
Prior work advanced density prediction through regression forests, interactive counting, and deep networks that process image regions fully convolutionally.
Other work addressed noisy crowd-sourced point annotations and filtered erroneous background predictions through segmentation.
Receptive-field counting showed that networks can learn object-identifying features while counting, which this paper extends by performing the task redundantly.
Proximity maps smooth cell predictions with cone-shaped distributions, but remain closer to density maps than count maps.

3. Fully Convolutional Redundant Counting

The paper frames counting as predicting object counts in small receptive fields over sparsely labeled, large images. Its fully convolutional pipeline pads the image, produces a count map, and combines local counts into an image-level total.

3.1. Problem Statement: The task is to count often-small objects in large images from few point-annotated training examples because labeling is labor-intensive.
3.2. Overview of Technique: The approach combines receptive-field counting, density-map ideas, and fully convolutional processing.
3.2. Overview of Technique: A smaller network scans the image to produce a count map whose entries represent object counts within a specific r × r receptive field.
3.2. Overview of Technique: The pipeline pads the image, processes it fully convolutionally, and combines local counts into the image’s total count.
3.2. Overview of Technique: Fully convolutional processing reduces overfitting through fewer parameters and more training examples obtained by splitting images into regions.

3.3. Input

The target image is built from point annotations by summing objects within each receptive field, with alignment between network outputs and regression targets required.

The target image T must align with F(I) so each receptive field matches the proper regression target.
Point annotations provide a simpler target representation than drawing object boundaries for segmentation.
For each output location, R(x, y) identifies the corresponding receptive-field pixel locations.
T[x, y] sums the annotated cells inside an r × r receptive field and becomes the regression target for that image region.

3.4. Fully Convolutional Redundant Counting

The fully convolutional network maps a 320 × 320 image to receptive-field count predictions using the Count-ception architecture, which avoids standard stride-based downsampling.

A 32 × 32 receptive field produces a 287 × 287 output on a 320 × 320 image, with each output pixel representing a target count.
Count-ception adapts Inception units with 1x1 and 3x3 convolutions while preserving tensor size across multiple layers.
The architecture replaces max pooling and stride=2 convolutions with large convolutions for downsampling.
Batch Normalization after every convolution is necessary for the model to train.

3.5. Loss Functions and Regularization

The method uses redundant receptive-field counts so averaging can reduce prediction errors, while L1 loss outperformed the tested alternatives and image-level loss encouraged over-fitting.

L1 loss performed best among the tested loss-function combinations.
Combining pixel-wise loss with an overall-image prediction loss caused over-fitting and did not assist training.
The model uses redundant counts to average over possible errors rather than relying on a single prediction.
The true count is recovered by dividing the sum of all pixel predictions by the number of redundant counts.
Redundant counting allows learning when point labels are off-center or outside a cell because the cell appears in the receptive field on average.

3.7. Limitations

Redundant counting gives up exact object coordinates and can produce correct overall counts without correctly identifying individual cells, especially in sparse training data.

The approach cannot localize each cell exactly with x, y coordinates, although count maps indicate where detections originated.
A correct overall count may result from average regression behavior rather than correctly identifying individual cells.
When training data contains many images without cells, the network may predict 0 to minimize the loss.
Training first on more balanced examples and then on sparser datasets is suggested as a possible solution.

4. Datasets

The evaluation uses synthetic bacterial-cell images, real bone-marrow images, and adipocyte histology regions. These datasets vary in imaging conditions, cell density, overlap, and object size, with adipocytes providing a particularly difficult counting case.

Dataset examples: Figure 6 presents examples of cells from each dataset used for evaluation.
VGG Cells: VGG Cells contains 200 simulated 256x256 bacterial-cell images with 174 ± 64 overlapping cells per image.The images simulate fluorescence-light microscopy at varied focal distances.
MBM Cells: MBM Cells is a modified bone-marrow dataset derived from eleven 1,200 × 1,200 images from eight healthy individuals.The images depict cell nuclei in blue and other constituents in pink and red; the passage also describes cropping them to 600 × 600.
Adipocyte Cells: Adipocyte Cells contains 200 regions of interest from human subcutaneous adipose tissue, downsampled from 1,700 × 1,700 windows to 150 × 150 images.The average cell count is 165±44.2, and adipocytes vary substantially in size.
Adipocyte Cells: Densely packed adjoining adipocytes with few gaps make this dataset a difficult test case for automated cell counting.

5. Experiments

Experiments evaluate Count-ception under a fixed-test protocol, compare it with segmentation and neural-network baselines, and examine how redundancy and model complexity affect performance. The experiments report that removing redundant counting reduces accuracy, while larger or simpler alternatives do not match the model’s performance.

Evaluation protocol: The evaluation fixes the testing-set size while varying training and validation sizes to simulate different numbers of labeled examples.Results are computed over at least 10 random splits and reported as mean and standard deviation.
Baselines: Count-ception is compared with Cell Profiler as a practitioner segmentation baseline and with Xie’s FCRN-A neural network.Only Count-ception and FCRN-A are neural-network-based approaches in this comparison.
Redundant counting: Matching Xie’s performance when redundant counting is disabled indicates that the reported advantage is associated with the redundant-counting configuration.The disabled condition is produced by changing the stride to eliminate redundant counting.
Implementation: The reported implementation uses Adam with a 0.005 learning rate, batch size 4, 1000 epochs, and Glorot initialization adjusted for ReLU gain.The best validation model is evaluated on the test set.
Redundant counting: A stride of 1, which provides the maximum redundant counting, is reported as optimal, while increasing the stride to the patch size reduces accuracy.
Model capacity: Increasing the receptive field to 64x64 did not improve performance, while larger models can add parameters that cause overfitting.The authors also report that shorter or narrower models lacked enough representational power, whereas wider models tended to overfit.
Runtime: The algorithm has nontrivial runtime because reducing parameters or depth did not preserve the same performance.The authors attribute significant performance to the complexity of the Inception modules.

6. Conclusion

Count-ception replaces density-map prediction with redundant counting to average errors and reduce overfitting, outperforming existing approaches while handling complicated cell structures. Its count map supports localization but does not easily provide object coordinates.

6. Conclusion: Redundant counting replaces density maps and averages predictions to reduce overfitting.The approach combines receptive-field counting with density-map ideas and fully convolutional processing.
6. Conclusion: Count-ception outperforms existing approaches and performs well with complicated cell structures, including adjoining cell walls.The paper describes the method as promising for tasks involving differently sized objects with complicated structure.
6. Conclusion: Table 2 evaluates test-set mean absolute error against prior work across datasets using repeated random splits and network initializations.Training and validation sizes vary while the held-out test-set size remains fixed.
6. Conclusion: Table 3 compares stride settings for reducing redundant counting using mean absolute error, including training-and-testing and test-only stride changes.Larger training strides expose the network to less data, while test-only strides reduce redundant predictions during evaluation.
6. Conclusion: The count map can support localization but cannot easily provide the x, y locations of individual objects.This limits the kind of object-level justification the method can directly provide.