关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
OpenI
/
aiforge
Not watched
33
71
Fork 75
Code
Releases 60 Wiki Activity
Issues 278 Pull Requests 0 Datasets Cloudbrain
Issues
/
Detail
Labels Milestones
New Issue

#4178 【大模型微调】同时部署的任务数量较多时，较大几率failed

Open
created 11 months ago by wangj · 2 comments
wangj commented 11 months ago

同时部署的任务数量较多时，较大几率FAILED。
包括两类：构建失败，部署失败。

同时部署的任务数量较多时，较大几率FAILED。 包括两类：构建失败，部署失败。
wangj added this to the V20230517 milestone 11 months ago
wangj added the
need review
label 11 months ago
wangj added the
bug
label 11 months ago
chenzh was assigned by wangj 11 months ago
chenzh commented 11 months ago
Owner

构建失败：构建当前无数量限制，多个同时构建有的应用会超时。云脑2人员排查说是云脑2服务并发问题，暂时不清楚并发上限，后续需要与云脑2跟进并增加限制同时创建AI应用的数量。
部署失败：部署数量已经限制上限为5个任务，还会存在资源不够的报错。经云脑2人员排查为自定义资源规格配置问题，已更改成默认资源规格。

构建失败：构建当前无数量限制，多个同时构建有的应用会超时。云脑2人员排查说是云脑2服务并发问题，暂时不清楚并发上限，后续需要与云脑2跟进并增加限制同时创建AI应用的数量。 部署失败：部署数量已经限制上限为5个任务，还会存在资源不够的报错。经云脑2人员排查为自定义资源规格配置问题，已更改成默认资源规格。
chenzh added the
test
label 11 months ago
wangj commented 11 months ago
Owner

受 #4191 影响，测试blocked.

受 #4191 影响，测试blocked.
wangj modified the milestone from V20230517 to V20230531 11 months ago
wangj modified the milestone from V20230531 to V20230628 10 months ago
wangj removed the
test
label 10 months ago
wangj removed this from the V20230628 milestone 10 months ago
Sign in to join this conversation.
finetune02
Clear branch or tag
Branches Tags
Labels
Apply labels
Clear labels
bug
Something is not working doc
需要更新帮助文档 duplicate
This issue or pull request already exists enhancement
enhancement feature
new feature frontend
前端开发 grampus
依赖虎鲸解决 help wanted
Need some help invalid
Something is wrong need review question
More information is needed reviewed test
Need test wait
问题的解决依赖于第三方的改进 WIP wontfix
This won't be fixed 严重
major bug 轻微
minor bug 重新开启 面向社区开发者
No Label
bug
doc
duplicate
enhancement
feature
frontend
grampus
help wanted
invalid
need review
question
reviewed
test
wait
WIP
wontfix
严重
轻微
重新开启
面向社区开发者
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
 chenzh
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes
Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit