关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
OpenI
/
aiforge
Not watched
33
71
Fork 75
Code
Releases 60 Wiki Activity
Issues 279 Pull Requests 0 Datasets Cloudbrain
Issues
/
Detail
Labels Milestones
New Issue

#4443 【启智GPU训练】支持实时输出日志

Closed
created 9 months ago by wangj · 4 comments
wangj commented 9 months ago

启智集群GPU训练不能实时输出日志，智算集群可以。建议优化。

来自于 小白训练营
zeizei/OpenI_Learning#972 训练任务无法实时输出日志信息
zeizei/OpenI_Learning#153 启智GPU集群的日志功能
zeizei/OpenI_Learning#1009 【建议】启智训练任务的日志能够实时输出训练日志

启智集群GPU训练不能实时输出日志，智算集群可以。建议优化。 来自于 小白训练营 https://openi.pcl.ac.cn/zeizei/OpenI_Learning/issues/972 训练任务无法实时输出日志信息 https://openi.pcl.ac.cn/zeizei/OpenI_Learning/issues/153 启智GPU集群的日志功能 https://openi.pcl.ac.cn/zeizei/OpenI_Learning/issues/1009 【建议】启智训练任务的日志能够实时输出训练日志
wangj added the
enhancement
label 9 months ago
wangj commented 9 months ago
Owner

有时候，启智集群GPU训练任务 running时也有实时日志输出。

有时候，启智集群GPU训练任务 running时也有实时日志输出。
chenyifan01 referenced this issue from a commit 9 months ago
#4443 启智GPU实时日志
chenyifan01 self-assigned this 9 months ago
chenyifan01 added the
test
label 9 months ago
wangj was assigned by chenyifan01 9 months ago
wangj added this to the V20230718 milestone 9 months ago
wangj commented 9 months ago
Owner

创建新的启智GPU训练任务，就可以实时显示日志了。备注：上线之前就在running的历史任务，该fix不生效。

创建新的启智GPU训练任务，就可以实时显示日志了。备注：上线之前就在running的历史任务，该fix不生效。
wangj closed this issue 9 months ago
wangj reopened this issue 9 months ago
wangj commented 9 months ago
Owner

有个任务运行了4个半小时，现还在running中，日志页签空。
测试任务名：wjtes20230711-old-gpu-running

有个任务运行了4个半小时，现还在running中，日志页签空。 测试任务名：wjtes20230711-old-gpu-running
wangj commented 9 months ago
Owner

有个任务运行了4个半小时，现还在running中，日志页签空。
测试任务名：wjtes20230711-old-gpu-running

符合预期。这是历史任务。

> 有个任务运行了4个半小时，现还在running中，日志页签空。 > 测试任务名：wjtes20230711-old-gpu-running 符合预期。这是历史任务。
wangj closed this issue 9 months ago
Sign in to join this conversation.
No Branch/Tag Specified
Clear branch or tag
Branches Tags
Labels
Apply labels
Clear labels
bug
Something is not working doc
需要更新帮助文档 duplicate
This issue or pull request already exists enhancement
enhancement feature
new feature frontend
前端开发 grampus
依赖虎鲸解决 help wanted
Need some help invalid
Something is wrong need review question
More information is needed reviewed test
Need test wait
问题的解决依赖于第三方的改进 WIP wontfix
This won't be fixed 严重
major bug 轻微
minor bug 重新开启 面向社区开发者
No Label
bug
doc
duplicate
enhancement
feature
frontend
grampus
help wanted
invalid
need review
question
reviewed
test
wait
WIP
wontfix
严重
轻微
重新开启
面向社区开发者
Milestone
Set milestone
Clear milestone
No items
No Milestone
V20230718
Assignees
Assign users
Clear assignees
No Assignees
 chenyifan01
 wangj
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes
Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit