Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

27 November 2018

Accepted: frontiers in Neuroanatomy, 2018

Authors: Chi Xiao1,2, Xi Chen1, Weifu Li 3, Linlin Li 1, Lu Wang4, Qiwei Xie1,5* and Hua Han1,2,6*

1 Institute of Automation, Chinese Academy of Sciences, Beijing, China,

2 School of Future Technology, University of Chinese Academy of Sciences, Beijing, China,

3 Faculty of Mathematics and Statistics, Hubei University, Wuhan, China,

4 Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China,

5 Data Mining Lab, Beijing University of Technology, Beijing, China,

6 Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China

Abstract: Recent studies have supported the relation between mitochondrial functions and degenerative disorders related to aging, such as Alzheimer’s and Parkinson’s diseases. Since these studies have exposed the need for details and high-resolution analysis of physical alterations of mitochondrial. However, due to the variety of mitochondrial structures, automated mitochondria segmentation and reconstruction in electron microscopy(EM) images have proven to be a difficult and challenging task.

This paper puts forward an effective and automated pipeline based on deep learning to realize mitochondria segmentation in different EM images. The proposed pipeline consists of three parts: (1) utilizing image registration and histogram equalization as image pre-processing steps to maintain the consistency of dataset; (2) proposing an effective approach for 3D mitochondria segmentation based on a volumetric, residual convolutional and deeply supervised network; and (3) employing a 3D connection method to obtain the relationship of mitochondria and displaying the 3D reconstruction results. To our knowledge, we are the first researchers to utilize a 3D fully residual convolutional network with a deeply supervised strategy to improve the accuracy of mitochondria segmentation.

The experimental results on anisotropic and isotropic EM volumes demonstrate the effectiveness of our method, and the Jaccard index of our segmentation(91.8% in anisotropy, 90.0% in isotropy) and F1 score of detection(92.2% in anisotropy, 90.9% in isotropy) suggest that our approach achieved state-of-the-art results. Our fully automated pipeline contributes to the development of neuroscience by providing neurologists with a rapid approach for obtaining rich mitochondria statistics and helping them elucidate the mechanism and function of mitochondria.

摘要: 最近的研究支持线粒体功能与衰老相关的退行性疾病(如阿尔茨海默症和帕金森病)之间的关系. 这些研究揭示了对线粒体物理改变的细节和高分辨率分析的需求. 然而,由于线粒体结构的多样性,电子显微镜(EM)图像中的自动线粒体分割和重建已被证明是困难且具有挑战性的任务.

本文提出了一种基于深度学习的有效自动化管道,来实现不同EM图像中的线粒体的分割. 提出的管道包括三个部分: (1) 利用图像配准和直方图均衡来预处理图像,保持数据的一致性; (2)提出一种基于Volume, residual卷积和深度监督网络的三维线粒体分割的有效方法; (3) 采用三维连接方法获得线粒体关系并显示三维重建结果. 据我们所知,我们是第一批利用3D全卷积residual卷积网络和深度监督策略提高线粒体分割准确性的研究人员.

各向异性和各相同性的EM volumes的试验结果证明了我们方法的有效性. 我们分割的Jaccard指数(各向异性为91.8,各相同性为90.0%)和检测的F1分数(各向异性为92.2, 各相同性为90.9%)表明我们方法取得了最先进的结果. 我们全自动化管道通过为神经科医生提供快速获取丰富线粒体统计分析数据并帮助他们阐明线粒体的机制和功能,为神经科学的发展做出了贡献.


In this work, we combined the residual block with variant 3D U-Net to extend and deepen the network. There are three advantages to our proposed architecture: (1) the 3D convolution network is capable of exploiting the 3D spatial information from volumetric EM data, (2) the variant U-net structure residual convolutional module could better extract features, and (3) injected auxiliary classifier layers could help avoid vanishing gradients. Figure 3 is a pictorial illustration of the proposed network.

As a variant of the U-net, this network consists of a contracting path and an expansive path. The contracting path contains 13 convolutional layers, and the expansive path contains 15 convolutional layers(我数着也是13个??); thus the whole network is not entirely symmetric.

Several summation-based skip connections were utilized to incorporate global information from higher layers and local cues from lower layers. Compared to concatenation-based skip connections, summation-based skip connections fused multilevel contextual more throughly and helped handle the vanishing gradient problem.

利用几个基于求和的skip connections来合并来自高层的全局信息和来自较底层的局部线索. 与基于concatenation的skip connection相比, 基于求和的skip connections更加完整地融合了多级上下文,并帮助处理消失的梯度问题.

Deeply Supervised Strategy

Due to the problem of vanishing gradients, it is challenging to train such a deep 3D network directly. Motivated by previous studies, we utilized a deeply supervised strategy by using injected supervision to train the network.

由于梯度消失的问题,直接训练这样深的3D网络是具有挑战性的。在之前研究的推动下,我们采用了一种深度监督的策略,通过injected supervision来训练网络.

As shown in Figure 3, several upsampling layers(with a stride of 2 x 2 x 1) were inserted into the hidden layers of the network, followed by auxiliary classifier layers. During the training process, the loss of auxiliary classifier layers wea added to the total loss of the total loss of the network with a discount weight(the weight values of the auxiliary losses were 0.15 and 0.3). During the testing process, these auxiliary networks were discarded.

The total cross-entropy loss function is defined as:

Evaluation Methodology

  • Jaccard index. This metric, which is also known as the VOC score, calculates the pixel-wise overlap between the ground truth(Y) and segmentation results(X).

  • Dice coefficient. This metric, which is similar to the Jaccard index, compares the similarity between the ground truth(Y) and segmentation results(X).

  • Conformity coeffcient. It is a global similarity coefficient, which is more sensitive and rigorous than the Jaccard index and Dice coefficient due to its better discriminiation capabilities.

  • Precision and recall. These metrics are related to the mitochondria counts in 3D. Precision is the probability that the dectected mitochondria are true, and the the recall is the probability that the true mitochondrial are successfully detected.

  • F1 score. Since precision and recall are often contradictory, this metirc is the weighted average of precision and recall, which shows the comprehensive performance.

关键点: 3D 卷积, residual connection, deeply supervised strategy(类似于FPN)