The question

Neuron morphological reconstruction is crucial for understanding brain functions, yet handling the massive datasets from advanced imaging technologies, like fluorescence micro-optical sectioning tomography (fMOST) ¹ and serial two-photon tomography ², is challenging. Traditional manual reconstruction, aided by tools like Fast Neurite Tracer (FNT) ³, has enabled whole-brain mapping projects such as the reconstruction of over 6,000 neurons in the mouse prefrontal cortex ³. However, manual reconstruction at the whole-brain level is too laborious to keep up with the rapid pace of imaging data generation.

Fully automatic methods developed for neuron reconstruction often lack accuracy, for long-range axons notably, where a single reconstruction error can result in the loss of entire axon branches. Thus, combining manual proofreading with automatic reconstruction is essential. However, existing tools like TeraVR ⁴ and Janelia Workstation ⁵, which have limited support for concurrent participation, are inadequate for large-scale collaboration over the Internet. This bottleneck highlights the need for better tools to support collaborative efforts in neuron reconstruction.

The solution

We developed Gapr for the projects of large-scale collaborative neuron reconstruction. Gapr is composed of several functional modules: gather, convert, trace, proofread, and fix (Fig. 1). The gather module performs imaging and tracing data collection and management for collaborative reconstruction. The convert module processes and uploads compressed imaging data, supporting on-demand conversion. The trace module conducts automatic reconstruction and automatic proofreading, generating preliminary results that are later reviewed by human annotators using the proofread and fix modules. Nodes and links represent continuous neuron structures, with attributes to track proofreading progress. The proofreading module allows annotators to correct local reconstruction errors and the fix module allows experienced annotators to resolve issues with more complexity, ensuring accurate neuron reconstruction.

As a preprocessing step, we used deep 3D U-Net ⁶ to segment imaging data, improving reconstruction accuracy over raw images. Overall reconstruction accuracy is enhanced by U-Net, with a false discovery rate less than 5% and a false-negative rate between 10% and 30% due to weak signals in the data. We utilized neuTube ⁷ to extract neurite signals from preprocessed images. To accommodate large datasets, reconstruction is performed in small cubes and integrated into the whole brain. Furthermore, Gapr offers automatic proofreading with a deep 3D residual network (ResNet) ⁸ for classification. High-confidence nodes can be skipped during manual proofreading, allowing human annotators to focus on low-confidence nodes. This substantially reduces manual proofreading load, with expected improvements as imaging quality advances.

In Gapr, multiple annotators can collaboratively proofread a shared reconstruction dataset. Annotators validate and process edits based on the latest version of reconstruction results fetched from the gather module to prevent inconsistencies, and receive real-time notifications of any rejected edits. Valid edits are immediately saved to the database and can be replayed to recover historical states. The server efficiently processes requests for the same dataset sequentially, ensuring a smooth collaborative experience. We found that, with a 2.50 GHz CPU and 1 MiB bandwidth, more than 100 annotators can be accommodated in Gapr to edit the same dataset concurrently with minimal delay. Furthermore, if users are distributed across multiple datasets, Gapr with a multiprocessing server is suitable for thousands of annotators to concurrently participate in the reconstruction.

Compared with other neuron reconstruction tools like Vaa3D ⁴ and Janelia Workstation ⁵, Gapr is well integrated with on-demand data conversion, automatic reconstruction, automatic proofreading and real-time large-scale collaborative neuron reconstruction. Accordingly, Gapr achieves a higher per-annotator proofreading speed on manual reconstruction and restricted proofreading, and a faster speed for massive neuron reconstruction in the whole-brain scale. Additionally, Gapr features a user-friendly interface, mobile device support, easy deployment, low hardware requirements and comprehensive reconstruction history. We previously reconstructed 6357 neurons from a total of 161 mouse brain samples using FNT ³. In comparison, utilizing Gapr, we reconstructed 4,278 neurons from 15 fMOST mouse brain datasets, thus yielding a much higher number of neurons per dataset. Our subsequent analysis of these Gapr-reconstructed neurons revealed biological insights into the morphological diversity of cortical interneurons and hypothalamic neurons in mouse. In sum, Gapr is a more efficient tool for large-scale neuron reconstruction.

Future directions

Gapr accelerates neuron reconstruction by incorporating deep learning for automatic reconstruction and proofreading, substantially reducing manual effort. Its user-friendly interface and internet-based collaboration boost reconstruction speed and overall proofreading throughput. Gapr's consecutive automatic reconstruction and automatic proofreading simplified reconstruction process, with unified node identifiers and avoiding the need for merging different versions of reconstruction, thus ensuring faster and more accurate proofreading at later stage. The reconstruction system of Gapr requires a small number of expert annotators, with crowd users handling most manual proofreading work, allowing the experts to focus on solving the complex structures. However, challenges of neuron reconstruction remain when facing the data of low image quality and high density of neuron labeling, hindering the effectiveness of automatic proofreading. Improving the training data for ResNet and developing advanced methods to distinguish complex structures such as branches and crossovers are necessary for the future improvement. Additionally, better imaging quality and labeling strategies are crucial to make the fixing stage more efficient.

An interesting future direction is to integrate the convert module of Gapr into the imaging pipeline to enable neuron reconstruction during imaging. Annotators can then guide on-demand imaging to speed up the process. The capability with terabyte-scale datasets of Gapr shows its potential for petabyte-scale data, paving the way to the single-neuron projectome reconstruction in primate brains.

References

[1]   Zheng, T., et al., Visualization of brain circuits using two-photon fluorescence micro-optical sectioning tomography. Opt Express, 2013. 21(8): p. 9839-50.

[2]   Economo, M.N., et al., A platform for brain-wide imaging and reconstruction of individual neurons. Elife, 2016. 5: p. e10566.

[3]   Gao, L., et al., Single-neuron projectome of mouse prefrontal cortex. Nat Neurosci, 2022. 25(4): p. 515-529.

[4]   Wang, Y., et al., TeraVR empowers precise reconstruction of complete 3-D neuronal morphology in the whole brain. Nat Commun, 2019. 10(1): p. 3474.

[5]   Winnubst, J., et al., Reconstruction of 1,000 Projection Neurons Reveals New Cell Types and Organization of Long-Range Connectivity in the Mouse Brain. Cell, 2019. 179(1): p. 268-281 e13.

[6]   Nassir Navab, J.H., William M. Wells, Alejandro F. Frangi, U-Net: convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. 2015, Springer International Publishing. p. 234-241.

[7]   Feng, L., T. Zhao, and J. Kim, neuTube 1.0: A New Design for Efficient Neuron Reconstruction Software Based on the SWC Format. eNeuro, 2015. 2(1).

[8]   Kaiming He, X.Z., Shaoqing Ren, Jian Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016, IEEE. p. 770-778.