Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
suhaibo666 caf8485beb | 1 month ago | |
---|---|---|
.github/workflows | 1 month ago | |
doc | 6 months ago | |
mindpet | 1 month ago | |
requirements | 1 month ago | |
test | 1 month ago | |
.flake8 | 7 months ago | |
.gitignore | 10 months ago | |
.pre-commit-config.yaml | 6 months ago | |
LICENSE | 1 year ago | |
README.md | 1 month ago | |
pylintrc | 6 months ago | |
set_up.py | 1 month ago |
MindPet(Pet:Parameter-Efficient Tuning)是属于Mindspore领域的微调算法套件。随着计算算力不断增加,大模型无限的潜力也被挖掘出来。但随之在应用和训练上带来了巨大的花销,导致商业落地困难。因此,出现一种新的参数高效(parameter-efficient)算法,与标准的全参数微调相比,这些算法仅需要微调小部分参数,可以大大降低计算和存储成本,同时可媲美全参微调的性能。
在代码仓根目录下运行以下命令,会生成dist文件夹以及whl包:
python set_up.py bdist_wheel
执行以下命令安装whl包:
pip install dist/mindpet-1.0.4-py3-none-any.whl
通过以下命令进行卸载:
pip uninstall mindpet
目前MindPet已提供以下六种经典低参微调算法以及一种提升精度的微调算法的API接口,用户可快速适配原始大模型,提升下游任务微调性能和精度;
微调算法 | 算法论文 | 使用说明 |
---|---|---|
LoRA | LoRA: Low-Rank Adaptation of Large Language Models | MindPet_DeltaAlgorithm_README 第一章 |
PrefixTuning | Prefix-Tuning: Optimizing Continuous Prompts for Generation | MindPet_DeltaAlgorithm_README 第二章 |
Adapter | Parameter-Efficient Transfer Learning for NLP | MindPet_DeltaAlgorithm_README 第三章 |
LowRankAdapter | Compacter: Efficient low-rank hypercom plex adapter layers | MindPet_DeltaAlgorithm_README 第四章 |
BitFit | BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models | MindPet_DeltaAlgorithm_README 第五章 |
R_Drop | R-Drop: Regularized Dropout for Neural Networks | MindPet_DeltaAlgorithm_README 第六章 |
P-Tuning v2 | P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks | MindPet_DeltaAlgorithm_README 第七章 |
MindPet支持用户根据 微调算法 或 模块名 冻结网络中部分模块,提供调用接口和配置文件两种实现方式。
使用说明参考MindPet_GraphOperation_README 第一章。
MindPet支持用户单独保存训练中可更新的参数为ckpt文件,从而节省存储所用的物理资源。
使用说明参考MindPet_GraphOperation_README 第二章。
MindPet(Pet:Parameter-Efficient Tuning)是属于Mindspore领域的微调算法套件
https://gitee.com/mindspore-lab/mindpet
Python Unity3D Asset Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》