Source-linked AI summary
CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy
Zekang Yang, Jiamin Li, Zhenghua Li, Jiaqi Fan, Zengcai Guo, Xiaolin Hu
TL;DR
Automatic neuron reconstruction is well developed on local blocks, but its structural accuracy and scalability across whole brains remain uncertain. CORAL addresses this gap with progressive local and brain-wide tasks, structure-aware metrics, and a whole-brain tracing framework; evaluations show that current methods still struggle with structurally correct, fully automated reconstruction.
Problem
Existing benchmarks mainly assess local image blocks and geometric similarity, leaving whole-brain topology and long-range continuity insufficiently evaluated.
Method
CORAL combines a curated whole-brain fMOST dataset, progressive block-level and brain-wide tasks, keypoint- and fiber-based metrics, and a framework extending local methods brain-wide.
Results
Current methods remain far from structurally correct and fully automated brain-wide reconstruction, while structure-aware metrics provide a more comprehensive assessment than geometry-based metrics.
Takeaways & Limitations
CORAL provides a standardized platform for developing and testing brain-wide neuron reconstruction methods.
Takeaways & Limitations
Brain-wide reconstruction remains constrained by break and merge errors that disrupt continuity, branching structure, or neuronal identity over long distances.
Abstract
from arXiv · showhide
Automatic neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved encouraging results on local image blocks, it remains unclear whether such progress translates to reconstruction that is both structurally accurate and scalable to the whole-brain scale. We present CORAL, the first benchmark for structure-aware evaluation of automatic neuron reconstruction from light microscopy images at both local and whole-brain scales. Built on a high-quality whole-brain fMOST dataset with carefully curated annotations, CORAL establishes two progressive tasks: block-level reconstruction, which evaluates reconstruction methods under limited spatial context, and brain-wide reconstruction, which assesses complete neuron reconstruction at the whole-brain scale. To account for topological correctness beyond geometric distance similarity, we introduce a structure-aware metric based on fiber prediction. To further achieve complete neuron reconstruction across the entire brain, we develop a brain-wide neuron tracing framework that extends arbitrary local reconstruction methods to the whole-brain scale through an iterative local-to-global process. Using this benchmark, we provide the first structure-aware comparison of mainstream methods for local neuron reconstruction and further evaluate their performance in brain-wide reconstruction. Our results underscore the importance of structure-aware evaluation and the need for more robust methods for complete neuron reconstruction.
1 Introduction
CORAL addresses gaps in evaluating neuron reconstruction beyond local geometric accuracy, introducing structure-aware assessment and brain-wide reconstruction settings.
- Research gap: Local benchmarks leave whole-brain structural accuracy and long-range continuity insufficiently evaluated.Many neurons extend beyond individual image blocks, leaving global topology and continuity underassessed.
- Research gap: Geometry-based metrics can miss biologically incorrect branching or long-range trajectories despite good point alignment.Neuron reconstruction is fundamentally a tree-structure prediction problem.
- Contributions: CORAL combines block-level and brain-wide tasks with keypoint- and fiber-based structure-aware metrics.The benchmark uses a curated whole-brain fMOST dataset and evaluates bifurcation, termination, and root-to-leaf fiber recovery.
- Contributions: A whole-brain tracing framework extends arbitrary local reconstruction algorithms to brain-wide reconstruction.This supports standardized large-scale evaluation of methods originally designed for local reconstruction.
- Findings: Experiments show that geometry-based metrics are insufficient and that current methods remain far from structurally correct, fully automated brain-wide reconstruction.Small local structural errors can propagate into large-scale whole-brain errors.
2 Related Work
Prior neuron-reconstruction datasets and metrics largely emphasize small local volumes and geometric or tree-based comparisons, leaving important whole-brain and diagnostic limitations.
- Datasets: DIADEM, BigNeuron, CWMBS, and NeuroFly primarily provide small-volume image-block benchmarks.Only one cited study evaluated complete neurons, and it did not establish a standard evaluation protocol.
- Reconstruction methods: Deep-learning approaches commonly enhance or segment neuronal structures before traditional reconstruction algorithms.This pipeline has generally improved performance relative to directly applying reconstruction methods.
- Metrics: DIADEM checks matched-node ancestry and path-length differences, but its complex protocol produces an abstract score.These design choices make inspecting failure cases and guiding algorithm improvement difficult.
- Metrics: DIADEM is unsuitable for many block-level settings and can spuriously count alternative same-length paths as correct.It assumes a directed tree rooted at a known soma, whereas local blocks may lack the soma and contain undirected graphs.
3 Dataset Description
CORAL uses curated whole-brain fMOST data with standardized annotations and broad neuron-morphology statistics to support reconstruction evaluation.
- Imaging: The fMOST pipeline includes dual-channel registration, stitching, artifact removal, and signal enhancement at 0.35µm × 0.35µm × 1µm resolution.The in-plane resolution is 0.35µm and the axial resolution is 1µm.
- Curation and annotation: The dataset contains 32 neurons annotated through three independent quality-control stages and exported in SWC format.The protocol includes initial annotation, independent review with re-annotation, and global cross-validation.
- Dataset statistics: The 32 neurons are distributed across hemispheres, with 15 somata in the left hemisphere and 17 in the right.Figure 2 visualizes their spatial distribution from multiple views.
- Dataset statistics: Neuron lengths range from 695 µm to 110,292 µm, while the dataset contains 1,421 dendritic fibers and 4,931 axonal fibers.Dendrite and axon length distributions are reported separately.
4 CORAL Benchmark
CORAL organizes evaluation into progressive block-level and brain-wide tasks, pairing a whole-brain tracing framework with structure-aware keypoint and fiber metrics.
- Benchmark design: CORAL enables local development on diverse blocks and brain-wide deployment through the whole-brain tracing framework.The framework addresses the impracticality of directly importing the full high-resolution brain volume.
- Whole-brain tracing: The whole-brain framework uses heuristic search, local tracing, and growing to iteratively expand reconstructions across overlapping blocks.It starts near the soma, selects leaf or boundary-near nodes, applies any local algorithm, and compares overlapping results.
- Evaluation split: The benchmark separates left-hemisphere training data from right-hemisphere testing data to prevent leakage.This hemisphere-based partition is used throughout the benchmark.
- Benchmark tasks: Block-level evaluation uses 1,904 cropped 300 × 300 × 300 blocks paired with SWC annotations.The blocks cover diverse morphology, sparsity, and signal intensity conditions.
- Benchmark tasks: Brain-wide evaluation reconstructs complete neurons from soma-initialized trees, including test neurons ranging from 1,404 µm to 86,680 µm.The task includes 746 dendrites and 2785 axons.
- Structure-aware metrics: The Keypoint metric separately evaluates bifurcations and leafs, while the Fiber metric evaluates root-to-leaf neurite recovery.Fiber matching uses overlap and an FIoU threshold η for valid matches.
- Structure-aware metrics: Compared with DIADEM, the Fiber metric is described as more intuitive, flexible, and comprehensive for reconstruction-quality characterization.It directly measures each neurite through root-to-leaf fibers.
5 Experiments and Results
Experiments compare reconstruction methods across block-level and brain-wide settings, showing that structure-aware metrics reveal failures missed by geometric scores and that long-range continuity remains difficult.
- Block-level Reconstruction: Ten segmentation models were combined with multiple reconstruction methods to evaluate performance and efficiency under block-level conditions.The study included state-of-the-art 3D segmentation models and neuron-specific curvilinear-structure models.
- Block-level Reconstruction: SegMamba combined with neuTube achieved the best structure-aware block-level reconstruction quality, with Fiber-F1 of 46.8.Reconstruction speeds differed substantially, spanning roughly three orders of magnitude; Kimimaro was fastest and NETracer slowest.
- Block-level Reconstruction: Better segmentation performance did not imply better reconstruction quality because segmentation and reconstruction methods were coupled through distinct error patterns.The clDice segmentation model had lower Dice than SegMamba and IVNet but achieved comparable or better reconstruction results in some cases.
- Block-level Reconstruction: Geometry-based metrics failed to characterize structural quality: Kimimaro led several geometric scores but lagged in Bifurcation-F1 and Fiber-F1, while neuTube and APP2 performed strongly on those structural metrics.Geometry-based metrics correlated with one another but showed no clear correlation with structure-based metrics.
- Brain-wide Reconstruction: End-to-end learning methods may reduce two-stage optimization difficulties, but SPE-DNR and NETracer showed limited long-range reconstruction ability, while SegMamba’s context modeling emphasized long-range information.The brain-wide evaluation used a local-to-global tracing framework and required more than 400 GPU-hours, with per-neuron runtimes from 30 seconds to 13.35 hours.
- Brain-wide Reconstruction: Brain-wide results showed structure-specific strengths: clDice with Kimimaro led dendrite reconstruction, SwinUNETR with APP2 led axon reconstruction, and SegMamba with Kimimaro achieved the best overall performance but only 35.1% Fiber-F1.Existing methods reconstructed dendrites more reasonably than long-range axonal structures, and no method was significantly superior for axon reconstruction.
- Brain-wide Reconstruction: Brain-wide errors included breaks, same-neuron crossover merges, and other-neuron crossover merges that can propagate local mistakes into missing branches, false positives, false negatives, or wasted tracing time.Reliable reconstruction must preserve continuity, branching structure, and neuronal identity over long distances.
6 Conclusion
CORAL provides a standardized platform for evaluating neuron reconstruction across block-level and brain-wide settings, combining structure-aware metrics with a local-to-global tracing framework. Its evaluation definitions and experiments support more reproducible assessment of geometric, topological, and whole-brain reconstruction performance.
- Benchmark and framework: CORAL unifies block-level and brain-wide reconstruction tasks, structure-aware metrics, and a whole-brain tracing framework for standardized evaluation.The framework is intended to support development and testing of brain-wide neuron reconstruction methods.
- Conclusion: The benchmark’s scope includes both local reconstruction under limited context and complete neuron reconstruction at whole-brain scale.This addresses the mismatch between cropped-volume benchmarks and neurons that extend beyond individual local volumes.
- Benchmark and framework: The whole-brain framework repeatedly searches for expansion candidates, traces local skeletons, and grows them into a neuron tree assembled from overlapping cube predictions.This local-to-global process begins from the soma and attaches newly traced structures to candidate nodes.
C.1 Segmentation Training and Testing Configuration
Segmentation models are trained and evaluated under a unified 3D-block protocol, while tracing is adapted to recover multiple disconnected neurite fragments within a local cube.
- Training randomly crops 128 × 128 × 128 blocks from annotated 300 × 300 × 300 cubes.
- Validation and testing use sliding-window inference on 300 × 300 × 300 cubes with 128 × 128 × 128 windows and 0.5 overlap.
- The models are optimized with AdamW using a learning rate and weight decay of 1 × 10−4.
- An iterative trace-and-remove strategy reruns tracing on residual images to reconstruct multiple disconnected neurite fragments.
D.1 Segmentation Results
Segmentation performance is reported for ten methods on the block-level test set.
- The test set compares segmentation performance across ten different methods.
D.2 Detailed Results on the Block-level Task
Block-level reconstruction methods are quantitatively evaluated using both keypoint-based and fiber-based metrics.
- The block-level task reports detailed quantitative results for different tracing methods using keypoint-based and fiber-based metrics.
D.3 Detailed Results on the Brain-wide Task
Brain-wide reconstruction results are reported with fiber-based metrics, including micro-F1 estimates and 95% BCa bootstrap confidence intervals.
- The brain-wide reconstruction task evaluates different tracing methods using fiber-based metrics.
- Table A.1 compares segmentation methods on the block-level task.
- The detailed block-level results identify best and second-best method performance using bold and underlined formatting, respectively.
- Whole-brain results report each value as a micro-F1 estimate followed by its 95% BCa bootstrap confidence interval from 10,000 resamples.
D.4 Parameter Sensitivity of Fiber Metric
Evaluation remains robust across stricter IoU thresholds, while out-of-domain testing and human comparisons assess generalization and practical performance. Whole-brain break and merge errors can produce extensive downstream omissions or false positives.
- Parameter sensitivity: Performance generally decreases as the IoU threshold η increases, while method rankings remain largely consistent across thresholds.The results indicate robustness to variations in η despite stricter overlap requirements.
- Out-of-domain evaluation: C-166 provides an independent out-of-domain whole-brain test containing 15 neurons for models trained on CORAL.All models were evaluated on this independently prepared and annotated sample.
- Human comparison: Professional annotators show relatively small inter-annotator variation, whereas current state-of-the-art methods remain substantially below human performance.Five annotators each labeled five neurons, with dendrite, axon, and overall Fiber performance reported in Table A.6.
- Whole-brain error modes: A break error in an axon can cause multiple downstream neurites to be missed, while merge errors generate false positives or trace into another neuron.The two merge-error patterns either redirect tracing toward the soma or into a different neuron.
F Limitations
CORAL is constrained by limited whole-brain annotations and mouse-only data, leaving broader morphology, imaging, and cross-species generalizability unresolved.
- Data scope: The benchmark covers a limited number of fully annotated whole-brain neurons and may not represent the full diversity of morphologies and imaging conditions.Future extensions are expected to broaden data diversity, protocol coverage, and evaluation criteria.
- Cross-species scope: Because all data come from mice, CORAL’s generalizability to other species remains unestablished.The benchmark has not yet been evaluated across species.