谢昕辰 c448646a92 | 1 year ago | |
---|---|---|
.. | ||
README.md | 1 year ago | |
mask2former_r50_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_r50_8xb2-160k_ade20k-512x512.py | 1 year ago | |
mask2former_r101_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_r101_8xb2-160k_ade20k-512x512.py | 1 year ago | |
mask2former_swin-b-in1k-384x384-pre_8xb2-160k_ade20k-640x640.py | 1 year ago | |
mask2former_swin-b-in22k-384x384-pre_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_swin-b-in22k-384x384-pre_8xb2-160k_ade20k-640x640.py | 1 year ago | |
mask2former_swin-l-in22k-384x384-pre_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_swin-l-in22k-384x384-pre_8xb2-160k_ade20k-640x640.py | 1 year ago | |
mask2former_swin-s_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_swin-s_8xb2-160k_ade20k-512x512.py | 1 year ago | |
mask2former_swin-t_8xb2-90k_cityscapes-512x1024.py | 1 year ago | |
mask2former_swin-t_8xb2-160k_ade20k-512x512.py | 1 year ago | |
metafile.yaml | 1 year ago |
Masked-attention Mask Transformer for Universal Image Segmentation
Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO) and semantic segmentation (57.7 mIoU on ADE20K).
pip install "mmdet>=3.0.0rc4"
Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
---|---|---|---|---|---|---|---|---|---|---|
Mask2Former | R-50-D32 | 512x1024 | 90000 | 5.67 | 9.17 | A100 | 80.44 | - | config | model | log |
Mask2Former | R-101-D32 | 512x1024 | 90000 | 6.81 | 7.11 | A100 | 80.80 | - | config | model | log) |
Mask2Former | Swin-T | 512x1024 | 90000 | 6.36 | 7.18 | A100 | 81.71 | - | config | model | log) |
Mask2Former | Swin-S | 512x1024 | 90000 | 8.09 | 5.57 | A100 | 82.57 | - | config | model | log) |
Mask2Former | Swin-B (in22k) | 512x1024 | 90000 | 10.89 | 4.32 | A100 | 83.52 | - | config | model | log) |
Mask2Former | Swin-L (in22k) | 512x1024 | 90000 | 15.83 | 2.86 | A100 | 83.65 | - | config | model | log) |
Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
---|---|---|---|---|---|---|---|---|---|---|
Mask2Former | R-50-D32 | 512x512 | 160000 | 3.31 | 26.59 | A100 | 47.87 | - | config | model | log) |
Mask2Former | R-101-D32 | 512x512 | 160000 | 4.09 | 22.97 | A100 | 48.60 | - | config | model | log) |
Mask2Former | Swin-T | 512x512 | 160000 | 3826 | 23.82 | A100 | 48.66 | - | config | model | log) |
Mask2Former | Swin-S | 512x512 | 160000 | 3.74 | 19.69 | A100 | 51.24 | - | config | model | log) |
Mask2Former | Swin-B | 640x640 | 160000 | 5.66 | 12.48 | A100 | 52.44 | - | config | model | log) |
Mask2Former | Swin-B (in22k) | 640x640 | 160000 | 5.66 | 12.43 | A100 | 53.90 | - | config | model | log) |
Mask2Former | Swin-L (in22k) | 640x640 | 160000 | 8.86 | 8.81 | A100 | 56.01 | - | config | model | log) |
Note:
ResNet
rather than ResNetV1c
.@inproceedings{cheng2021mask2former,
title={Masked-attention Mask Transformer for Universal Image Segmentation},
author={Bowen Cheng and Ishan Misra and Alexander G. Schwing and Alexander Kirillov and Rohit Girdhar},
journal={CVPR},
year={2022}
}
@inproceedings{cheng2021maskformer,
title={Per-Pixel Classification is Not All You Need for Semantic Segmentation},
author={Bowen Cheng and Alexander G. Schwing and Alexander Kirillov},
journal={NeurIPS},
year={2021}
}
No Description
Python Markdown Shell Dockerfile other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》