English | 简体中文
Brief Introduciton
3D medical data annotation is an important part of training 3D image segmentation models and promotes disease diagnosis and treatment prediction, but 3D medical data annotation relies on time-consuming and laborious manual annotation by professionals. The low labeling efficiency leads to the lack of large-scale labeling data, which seriously hinders the development of medical AI. To solve this problem, we launched EISeg-Med3D, an intelligent annotation platform for 3D medical images based on interactive segmentation.
EISeg-Med3D is a 3D slicer extension for performing Efficient Interactive Segmentation on Medical image in 3D medical images. Users will guide a deep learning model to perform segmentation by providing positive and negative points. It is simple to install, easy to use and accurate, which can achieve ten times efficiency lift compares to manual labelling. At present, our medical annotation provides the try-on experience on the specified MRI vertebral data. If there is a need for 3D annotation on other data, you can make a contact.
Feature
-
Efficient:Each category only needs a few clicks to generate 3d segmentation results, ten times efficient compares to time-consuming and laborious manual annotation.
-
Accurate:The mIOU can reach 0.85 with only 3 clicks. with the segmentation editor equipped with machine learning algorithm and manual annotation, 100% accuracy is right on your hand.
-
Convenient:Install our plugin within three steps; labeling results and progress are automatically saved; the transparency of labeling results can be adjusted to improve labeling accuracy; user-friendly interface interaction makes labeling worry-free and hassle-free。
Contents
- News
- EISeg-Med3D Model Introduction
- User Guide
- TODO
- License
- Thanks
最新消息
- [2022-09] EISeg-Med3D is officially released, a user-friendly, efficient and intelligent 3D medical image annotation platform including high-precision models on specified vertebral data.
EISeg-Med3D Model
The EISeg-Med 3D model structure is shown in the figure below. We innovatively introduce the 3D model into the medical interactive segmentation, and modify the point sampler module and the click feature extrator of RITM to be compatible with 3D data, so as to directly label 3D medical images. Compared with 2D interactive annotation on 3D images, our method is more acurate and more efficient.
The overall model includes two parts: click generation module, click feature generation module, click feature and input image fusion and segmentation model:
- Click generation module 3D click sampler: generate the positive and negative click through sampling on the 3D labelled data directly, where the positive point is a random point at the center of the 3D target segment, and the negative point is a random point at the edge of the 3D target segment.
- Click feature generation module 3D feature extractor: After the click is generated, a sphere with radius R is generated on the basis of the original click in the form of disk to expand the feature coverage.
- Fusion of click feature and input image: The input image and the generated click feature are remapped and fused through the convolution block, so that the network obtains information from both the image and the clicks, and performs 3D segmentation on a the assigned area to obtain the annotation result.
- Segmentation model: The segmentation model is the 3D segmentation model Vnet, generates the prediction results shown in the figure, and approximates the real results under the constraints of Dice loss and CE loss. Thus, in the prediction stage, input images and specified clicks generate the desired annotation results.
EISeg-Med3D Model
User Guide
The overall process of using EISeg-Med3D is shown in the figure below. We will introduce in the following three steps including environment installation, model and data download and user guide. The steps to use our platform can also be found in the video in the introduction.
The overall process
The process of AI-assisted labelling
环境安装
-
Download and install 3D slicer:Slicer website
-
Download code of EISeg-Med3D:
git clone https://github.com/PaddlePaddle/PaddleSeg.git
- Find the slicer python executor in python interpreter of slicer. Then install PaddlePaddle refer to install doc in windows cmd.
import sys
import os
sys.executable # "D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe"
In CMD, If you install on Windows with CUDA 11.1 GPU, follow the command here::
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip install paddlepaddle-gpu==2.3.1.post111 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
Common FAQ
- FileNotFoundError when install PaddlePaddle:
Solution:Find the subprocess.py that raise the error, change the attribute of Popen shell=True。
- ERROR: No .egg-info directory found in xxx:
Solution:Please refer to https://github.com/PaddlePaddle/PaddleSeg/issues/2718. Execute the following code will solve the error.
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip uninstall setuptools
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip install paddleseg simpleitk
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip install setuptools
- When load module, the poped window says "One or more requested modules and/or depandencoes may not have been loaded".
Solution:You may forget to install some library we need to import, eg: paddle, paddleseg, simpleitk and etc. Please refer to step 1 and 2 to install and restart Slicer to import.
- Fail to open extention: xxx/PaddleSeg/EISeg/med3d/EISefMed3D
Solution:The path you need to choose is "xxx/PaddleSeg/EISeg/med3d/" but not "xxx/PaddleSeg/EISeg/med3d/EISefMed3D"
Model and Data Downloading
Currently we provide trial experience on the following models and data:
User Guide
1. Load the Extension
- Click on "Select Extension",and choose PaddleSeg/EISeg/med3d directory, and click to load corresponding module.
* After loading, switch to EISegMed3D。
2. Load Model
- Load the downloaded model in
Model Settings
:
click on the ...
button of Model Path
, choose local file of .pdodel
suffix and load .pdiparams
file in Param Path
in the same way.
- Click on
Load Static Model
button. And Sucessfully loaded model to gpu!
window will be prompt is the model is loaded successfully.
3. Load Medical Data
- Click on the button behind
Data Folder
, choose the folder that you saved your downloaded data. And all of the data under that folder will be loaded and you can see the labelling status of loaded data in Progress
.
4. Switch Between Images.
- Click on the
Prev Scan
button to see the previous image.
- Click on the
Next Scan
button to see the next image.
5. Start to label
- Click
Add/Remove
in Segment Editor
to add or remove the label. You can change the name of added label by double click the label item.
- Choose the label you want to label and click on the
Positive Point
or Negative Point
to enter interactive label mode。
- Click on
Finish Segment
button to finish annotation of current segment, you can further edit the annotatioin using tools in segment editor or you can repeat previous step to label next category. If you finished the annotation on this case, you can click on the Finish Scan
button.
6. Check Label Progress
- In
Annotation Progress
of Progress
, you can checkout the labelling progress of loaded images.
- Doule click on one of the image in the chart below
Annotation Progress
will jump to the corresponding image.
TODO
In the future, we want to continue to develop EISeg-Med3D in these aspects, welcome to join our developer team.
License
EISeg-Med3D is released under the Apache 2.0 license.
Attribution
Thanks to Idea icons created by Vectors Market - Flaticon for facsinating icons.