Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
janedx b3209485e6 | 1 year ago | |
---|---|---|
Fig1.svg | 1 year ago | |
README.md | 1 year ago | |
TFIDF.py | 1 year ago | |
config.py | 1 year ago | |
extract_label.py | 1 year ago | |
iu_xray.py | 1 year ago | |
knowledge_distiller.py | 1 year ago | |
models.py | 1 year ago |
This repository contains the code accompanying the paper "A Self-Guided Framework for Radiology Report Generation", accepted for publication at MICCAI 2022 (https://arxiv.org/abs/2206.09378).
If you find this repo useful for your research, please consider citing our paper:
@article{2206.09378,
Author = {Jun Li and Shibo Li and Ying Hu and Huiren Tao},
Title = {A Self-Guided Framework for Radiology Report Generation},
Year = {2022},
Eprint = {arXiv:2206.09378},
}
Automatic radiology report generation is essential to computer-aided diagnosis. Through the success of image captioning, medical report generation has been achievable. However, the lack of annotated disease labels is still the bottleneck of this area. In addition, the image-text data bias problem and complex sentences make it more difficult to generate accurate reports. To address these gaps, we present a self-guided framework (SGF), a suite of unsupervised and supervised deep learning methods to mimic the process of human learning and writing. In detail, our framework obtains the domain knowledge from medical reports without extra disease labels and guides itself to extract fined-grain visual features associated with the text. Moreover, SGF successfully improves the accuracy and length of medical report generation by incorporating a similarity comparison mechanism that imitates the process of human self-improvement through comparative practice. Extensive experiments demonstrate the utility of our SGF in the majority of cases, showing its superior performance over state-of-the-art methods. Our results highlight the capacity of the proposed framework to distinguish fined-grained visual details between words and verify its advantage in generating medical reports.
To run Knowledge Distiller
cd Knowledge_Distiller
python knowledge_distiller.py
To train the SGF
cd KMVE_RG
python my_main.py
To evaluate the model
cd KMVE_RG
python my_predict.py
To change config of the network
You can change the config of the network in KMVE_RG/config.py
Our experiments are conducted on the IU-Xray public dataset. IU-Xray is a large-scale dataset of chest X-rays, constructed by 7,470 chest X-ray images and 3,955 radiology reports. You can download the images' file through https://openi.nlm.nih.gov/.
cudatoolkit=11.1.1
python=3.8.11
pytorch=1.9.0=py3.8_cuda11.1_cudnn8_0
scikit-learn=0.23.2
scipy=1.6.2
seaborn=0.11.2
yacs=0.1.8
yaml=0.2.5
This work was supported in part by Key-Area Research and Development Program of Guangdong Province (No.2020B0909020002), National Natural Science Foundation of China (Grant No. 62003330), Shenzhen Fundamental Research Funds (Grant No. JCYJ20200109114233670, JCYJ20190807170407391), and Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. This work was also supported by Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology. The implementation of Self-guided Framework is mainly based on R2Gen. We thanks the authors for their efforts.
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》