Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
Boyin Zhang 409caf8548 | 1 year ago | |
---|---|---|
.. | ||
configs | 1 year ago | |
models | 1 year ago | |
README.md | 1 year ago |
DEST: Depth Estimation with Simplified Transformer
Transformer and its variants have shown state-of-the-art results in many vision tasks recently, ranging from image classification to dense prediction. Despite of their success, limited work has been reported on improving the model efficiency for deployment in latency-critical applications, such as autonomous driving and robotic navigation. In this paper, we aim at improving upon the existing transformers in vision, and propose a method for Dense Estimation with Simplified Transformer (DEST), which is efficient and particularly suitable for deployment on GPU-based platforms. Through strategic design choices, our model leads to significant reduction in model size, complexity, as well as inference latency, while achieving superior accuracy as compared to state-of-the-art in the task of self-supervised monocular depth estimation. We also show that our design generalize well to other dense prediction task such as semantic segmentation without bells and whistles.
All the commands below rely on the correct configuration of PYTHONPATH
, which should point to the mmsegmentaions directory so that Python can locate the configuration files in mmsegmentation.
Preparing cityscapes
dataset following this Dataset Preparing Guide
mim train mmsegmentation projects/dest/configs/dest_simpatt-b0_1024x1024_160k_cityscapes.py --work-dir work_dirs/dest
To train on multiple GPUs, e.g. 8 GPUs, run the following command:
mim train mmsegmentation projects/dest/configs/dest_simpatt-b0_1024x1024_160k_cityscapes.py --work-dir work_dirs/dest --launcher pytorch --gpus 8
mim test mmsegmentation projects/dest/configs/dest_simpatt-b0_1024x1024_160k_cityscapes.py --work-dir work_dirs/dest --checkpoint ${CHECKPOINT_PATH} --eval mIoU
Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | mIoU | mIoU(ms+flip) | config | download |
---|---|---|---|---|---|---|---|---|---|
DEST | SMIT-B0 | 1024x1024 | 160000 | - | - | 64.34 | - | config | model | log |
DEST | SMIT-B1 | 1024x1024 | 160000 | - | - | 68.21 | - | config | model | log |
DEST | SMIT-B2 | 1024x1024 | 160000 | - | - | 71.89 | - | config | model | log |
DEST | SMIT-B3 | 1024x1024 | 160000 | - | - | 73.51 | - | config | model | log |
DEST | SMIT-B4 | 1024x1024 | 160000 | - | - | 73.99 | - | config | model | log |
DEST | SMIT-B5 | 1024x1024 | 160000 | - | - | 75.28 | - | config | model | log |
Note:
@article{YangDEST,
title={Depth Estimation with Simplified Transformer},
author={Yang, John and An, Le and Dixit, Anurag and Koo, Jinkyu and Park, Su Inn},
journal={arXiv preprint arXiv:2204.13791},
year={2022}
}
Milestone 2: Indicates a successful model implementation.
Milestone 3: Good to be a part of our core package!
Type hints and docstrings
Unit tests
Code polishing
Metafile.yml
Move your modules into the core package following the codebase's file hierarchy structure.
Refactor your modules into the core package following the codebase's file hierarchy structure.
No Description
Python Markdown Shell Dockerfile other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》