Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
LGOODSKY b1dd373ee4 | 1 year ago | |
---|---|---|
.. | ||
BiT.png | 1 year ago | |
BiT50_ascend.yaml | 1 year ago | |
README.md | 1 year ago | |
README_CN.md | 1 year ago | |
bit_resnet50_ascend.yaml | 1 year ago |
Transfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural
networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale
up pre-training, and propose a simple recipe that we call Big Transfer
(BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over
20 datasets. BiT performs well across a surprisingly wide range of data
regimes — from 1 example per class to 1 M total examples. BiT achieves
87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3%
on the 19 task Visual Task Adaptation Benchmark (VTAB). On small
datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class,
and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed
analysis of the main components that lead to high transfer performance.
Model | Context | Top-1 (%) | Top-5 (%) | Params(M) | Train T. | Infer T. | Download | Config | Log |
---|---|---|---|---|---|---|---|---|---|
BiT50-S | D910x8-G | 76.81 | 93.17 | 25 | 652s/epoch | 189.8ms/step | model | cfg | log |
Please refer to the installation instruction in MindCV.
Please download the ImageNet-1K dataset for model training and validation.
Hyper-parameters. The hyper-parameter configurations for producing the reported results are stored in the yaml files in mindcv/configs/BigTransfer
folder. For example, to train with one of these configurations, you can run:
# train BiT on 8 NPUs
mpirun -n 8 python train.py -c configs/BigTransfer/BiT50_ascend.yaml --data_dir /path/to/imagenet
Note that the number of GPUs/Ascends and batch size will influence the training results. To reproduce the training result at most, it is recommended to use the same number of GPUs/Ascends with the same batch size.
Detailed adjustable parameters and their default value can be seen in config.py
To validate the trained model, you can use validate.py
. Here is an example for BiT-50 to verify the accuracy of pretrained weights.
python validate.py -c configs/BigTransfer/BiT50_ascend.yaml --data_dir /path/to/imagenet --ckpt_path /path/to/ckpt
Please refer to the deployment tutorial in MindCV.
No Description
Jupyter Notebook Python Markdown Text
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》