Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
cc 4a2f39690c | 1 year ago | |
---|---|---|
.. | ||
configs | 2 years ago | |
datasets | 2 years ago | |
models | 1 year ago | |
scripts | 1 year ago | |
tools | 2 years ago | |
README.md | 1 year ago | |
export.py | 1 year ago | |
predict.py | 1 year ago | |
train.py | 3 years ago | |
val.py | 2 years ago |
The implementation of Hierarchical Multi-Scale Attention based on PaddlePaddle. [Paper]
Based on the above work, we made some optimizations:
We achieve mIoU of 87% on Cityscapes validation set.
The actual effect is as follows (for high-definition pictures, please click here).
System Requirements:
Highly recommend you install the GPU version of PaddlePaddle, due to large overhead of segmentation models, otherwise it could be out of memory while running the models. For more detailed installation tutorials, please refer to the official website of PaddlePaddle。
You should use API Calling method to install PaddleSeg for flexible development.
pip install paddleseg
Download following files and put into data/cityscapes
directory. Then unzip these files.
mkdir -p data/cityscapes
Firstly please download 3 files from Cityscapes dataset
Run the following commands to do the label conversion:
pip install cityscapesscripts
python ../../tools/data/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
Where 'cityscapes_path' should be adjusted according to the actual dataset path. 'num_workers' determines the number of processes started and the size can be adjusted according to the actual situation.
Then download and uncompress Autolabelled-Data from google drive
Delete useless tmp
directory in refinement_final
directory.
rm -r tmp/
Convert autolabelled data according to PaddleSeg data format:
python tools/convert_cityscapes_autolabeling.py --dataset_root data/cityscapes/
Finally, you need to organize data following the below structure.
cityscapes
|
|--leftImg8bit
| |--train
| |--val
| |--test
|
|--gtFine
| |--train
| |--val
| |--test
|
|--leftImg8bit_trainextra
| |--leftImg8bit
| |--train_extra
| |--augsburg
| |--bayreuth
| |--...
|
|--convert_autolabelled
| |--augsburg
| |--bayreuth
| |--...
mkdir -p saved_model && cd saved_model
wget https://bj.bcebos.com/paddleseg/dygraph/cityscapes/mscale_ocr_hrnetw48_cityscapes_autolabel_mapillary/model.pdparams
cd ..
Model | Backbone | mIoU | mIoU (flip) | mIoU (5 scales + flip) |
---|---|---|---|---|
MscaleOCRNet | HRNet_w48 | 86.89% | 86.99% | 87.00% |
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -u -m paddle.distributed.launch val.py \
--config configs/mscale_ocr_cityscapes_autolabel_mapillary.yml --num_workers 3 --model_path saved_model/model.pdparams
The reported mIoU should be 86.89. This evaluates with scales of 0.5, 1.0 and 2.0. This requires 14.2GB of GPU memory.
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -u -m paddle.distributed.launch val.py \
--config configs/mscale_ocr_cityscapes_autolabel_mapillary.yml --num_workers 3 --model_path saved_model/model.pdparams \
--aug_eval --flip_horizontal
The reported mIoU should be 86.99. This evaluates with scales of 0.5, 1.0, 2.0 and flip horizontal. This requires 14.2GB of GPU memory.
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -u -m paddle.distributed.launch val.py \
--config configs/mscale_ocr_cityscapes_autolabel_mapillary_ms_val.yml --num_workers 3 --model_path saved_model/model.pdparams \
--aug_eval --flip_horizontal
The reported mIoU should be 87.00. This evaluates with scales of 0.5, 1.0, 1.5, 2.0, 2.5 and flip horizontal. This requires 21.2GB of GPU memory.
mkdir -p pretrain && cd pretrain
wget https://bj.bcebos.com/paddleseg/dygraph/cityscapes/ocrnet_hrnetw48_mapillary/pretrained.pdparams
cd ..
Pretrained weights were obtained by pretraining on the Mapillary dataset from OCRNet (backbone is HRNet w48).
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -u -m paddle.distributed.launch train.py \
--config configs/mscale_ocr_cityscapes_autolabel_mapillary.yml --use_vdl \
--save_dir saved_model/mscale_ocr_cityscapes_autolabel_mapillary --save_interval 2000 --num_workers 5 --do_eval
Note that this requires 32GB of GPU memory. You can remove argument --do_eval
to turn off evaluation during training, thus it only requires 25GB of GPU memory.
If you run out of memory, try to lower the crop size.
Run the following command to export the inference model.
python export.py \
--config configs/mscale_ocr_cityscapes_autolabel_mapillary_ms_val.yml \
--save_dir ./output \
--input_shape 1 3 2048 1024
We can use the following deployment methods to deploy the inference model.
Platform | Library | Tutorial |
---|---|---|
Python | Paddle prediction library | e.g. |
C++ | Paddle prediction library | e.g. |
Mobile | PaddleLite | e.g. |
Front-end | PaddleJS | e.g. |
Other deployment documents:
Thanks for their contributions!
飞桨高性能图像分割开发套件,端到端完成从训练到部署的全流程图像分割应用。
https://github.com/PaddlePaddle/PaddleSeg
Python Markdown Text Shell Java other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》