Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
Myle Ott f34abcf2b6 | 2 years ago | |
---|---|---|
.. | ||
linformer_src | 2 years ago | |
README.md | 3 years ago |
This example contains code to train Linformer models as described in our paper
Linformer: Self-Attention with Linear Complexity.
You can mostly follow the RoBERTa pretraining README,
updating your training command with --user-dir examples/linformer/linformer_src --arch linformer_roberta_base
.
If you use our work, please cite:
@article{wang2020linformer,
title={Linformer: Self-Attention with Linear Complexity},
author={Wang, Sinong and Li, Belinda and Khabsa, Madian and Fang, Han and Ma, Hao},
journal={arXiv preprint arXiv:2006.04768},
year={2020}
}
No Description
Python Cuda C++ Cython Markdown other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》