MUSCLE - MICCAI 2022
这是一篇论文 "MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts" 的相关介绍。
该论文发布于MICCAI 2022。
简介
MUSCLE的主标是通过预训练一个主干网络,来提高深度学习在医学影像分析任务中的性能。
该论文的所有代码均使用PaddlePaddle框架实现。
框架
MUSCLE聚合了从不同人体部位收集的多个Xray图像数据集,并作用于各种Xray影像的分析任务。
我们提出了多数据集动量对比表征学习(MD-MoCo)模块和多任务持续学习模块,
以自我监督的持续学习方式对深度学习框架的主干网络进行预训练。
预训练的模型可以使用特定任务的head对目标任务进行微调,并取得极佳的性能。
数据集
Datasets |
Body Part |
Task |
Train |
Valid |
Test |
Total |
Only Used for the first step (MD-MoCo) of MUSCLE |
NIHCC |
Chest |
N/A |
112,120 |
N/A |
N/A |
112,120 |
China-Set-CXR |
Chest |
N/A |
661 |
N/A |
N/A |
661 |
Montgomery-Set-CXR |
Chest |
N/A |
138 |
N/A |
N/A |
138 |
Indiana-CXR |
Chest |
N/A |
7,470 |
N/A |
N/A |
7,470 |
RSNA Bone Age |
Hand |
N/A |
10,811 |
N/A |
N/A |
10,811 |
Used for all three steps of MUSCLE |
Pneumonia |
Chest |
Classification |
4,686 |
585 |
585 |
5,856 |
MURA |
Various Bone |
Classification |
32,013 |
3,997 |
3,997 |
40,005 |
Chest Xray Masks and labels |
Chest |
Segmentation |
718 |
89 |
89 |
896 |
TBX |
Chest |
Detection |
640 |
80 |
80 |
800 |
Total |
N/A |
N/A |
169,257 |
4,751 |
4,479 |
178,757 |
实验
实验设置
- 主干网络
- 医学影像分析任务
- 肺炎分类任务 (Pneumonia),
- 骨骼异常分类任务 (MURA)
- 肺部分割任务 (Lung)
- 结核病Bounding Box检测 (TBX)
- Head网络
- 分类任务:Fully-Connected (FC) Layer
- 分割任务:DeepLab-V3
- 检测任务:FasterRCNN
- 基线的预训练算法
- Scratch: 模型主干网络使用Kaiming’s initialization进行参数初始化
- ImageNet: 模型主干网络使用官方发布的ImageNet进行参数初始化
- MD-MoCo: 模型主干网络只使用在多数据源的Xray图像进行MoCo学习的参数进行初始化
- MUSCLE−−: 模型的初始化策略和MUSCLE一致,但是不采用我们设计的跨任务记忆与循环和重组学习计划模块
不同身体部位的Xray数据集的结果
注意:Pneumonia是由胸片图像构成的数据集,而MURA由骨骼图像构成
Datasets |
Backbones |
Pre-train |
Acc. |
Sen. |
Spe. |
AUC(95%CI) |
Pneumonia |
ResNet-18 |
Scratch |
91.11 |
93.91 |
83.54 |
96.58(95.09-97.81) |
ImageNet |
90.09 |
93.68 |
80.38 |
96.05(94.24-97.33) |
MD-MoCo |
96.58 |
97.19 |
94.94 |
98.48(97.14-99.30) |
MUSCLE-- |
96.75 |
97.66 |
94.30 |
99.51(99.16-99.77) |
MUSCLE |
97.26 |
97.42 |
96.84 |
99.61(99.32-99.83) |
ResNet-50 |
Scratch |
91.45 |
92.51 |
88.61 |
96.55(95.08-97.82) |
ImageNet |
95.38 |
95.78 |
94.30 |
98.72(98.03-99.33) |
MD-MoCo |
97.09 |
98.83 |
92.41 |
99.53(99.23-99.75) |
MUSCLE-- |
96.75 |
98.36 |
92.41 |
99.58(99.30-99.84) |
MUSCLE |
98.12 |
98.36 |
97.47 |
99.72(99.46-99.92) |
MURA |
ResNet-18 |
Scratch |
81.00 |
68.17 |
89.91 |
86.62(85.73-87.55) |
ImageNet |
81.88 |
73.49 |
87.70 |
88.11(87.18-89.03) |
MD-MoCo |
82.48 |
72.27 |
89,57 |
88.28(87.28-89.26) |
MUSCLE-- |
82.45 |
74.16 |
88.21 |
88.41(87.54-89.26) |
MUSCLE |
82.62 |
74.28 |
88.42 |
88.5o(87.46-89.57) |
RcsNet-50 |
Scratch |
80.50 |
65.42 |
90.97 |
86.22(85.22-87.35) |
ImngeNet |
81.73 |
68.36 |
91.01 |
87.87(86.85-88.85) |
MD-MoCo |
82.35 |
73.12 |
88.76 |
87.89(87.06-88.88) |
MUSCLE-- |
81.10 |
69.03 |
89.48 |
87.14(86.10-88.22) |
MUSCLE |
82.60 |
74.53 |
88.21 |
88.37(87.38-89.32) |
不同任务的结果
注意:Lung为肺部分割任务,而TBX为检测任务
Backbones |
Pre-train |
Lung |
TBX |
Dice |
mloU |
mAP |
AP-Active |
AP-Latent |
ResNet-18 |
Scratch |
95.24 |
94.00 |
30.71 |
56.71 |
4.72 |
ImageNet |
95.26 |
94.10 |
29.46 |
56.27 |
2.66 |
MD-MoCo |
95.31 |
94.14 |
36.00 |
67.17 |
4.84 |
MUSCLE-- |
95.14 |
93.90 |
34.70 |
63.43 |
5.97 |
MUSCLE |
95.37 |
94.22 |
36.71 |
64.84 |
8.59 |
ResNet-50 |
Scratch |
93.52 |
92.03 |
23.93 |
44.85 |
3.01 |
ImageNet |
93.77 |
92.43 |
35.61 |
58.81 |
12.42 |
MD-MoCo |
94.33 |
93.04 |
36.78 |
64.37 |
9.18 |
MUSCLE-- |
95.04 |
93.82 |
35.14 |
57.32 |
12.97 |
MUSCLE |
95.27 |
94.10 |
37.83 |
63.46 |
12.21 |
Citation
如果我们的项目在学术上帮助到你,请考虑以下引用:
@inproceedings{liao2022muscle,
title={MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts},
author={Weibin, Liao and Haoyi, Xiong and Qingzhong, Wang and Yan, Mo and Xuhong, Li and Yi, Liu and Zeyu, Chen and Siyu, Huang and Dejing, Dou},
booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
year={2022},
organization={Springer}
}