Deleting a branch is permanent. It CANNOT be undone. Continue?
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》
任务目标:
2022-06-24:
存在问题:
(1)1bit通信实现问题:继续参考DeepSpeed的实现方法进行
(2)目前设计的梯度量化方法中,m,v等中间变量采用的是每张卡本地的状态值,而不是全局的状态值,是否影响模型收敛还有待验证,下面是BinarySGD、TrinarySGD方法和DeepSpeed中1bit Adam的算法设计对比