关于云脑任务中统一路径访问方式的公告>>>
OpenI
/
octopus
Not watched
29
46
Fork 63
Code
Releases 34 Wiki Activity
Issues 122 Pull Requests 0 Datasets Cloudbrain
Issues
/
Detail
Labels Milestones
New Issue

#775 监控接口返回燧原、昇腾、寒武纪、天数等其他芯片的gpu使用率、显存使用率

Closed
created 1 year ago by linfj · 4 comments
linfj commented 1 year ago

可新增字段accCardUtil、accCardMemUsage

可新增字段accCardUtil、accCardMemUsage
wakinzhang was assigned by linfj 1 year ago
wakinzhang added the
测试中
label 1 year ago
wakinzhang added the
开发完成
label 1 year ago
yangxzh1 commented 1 year ago
Collaborator

天数的训练任务，监控接口返回的accCardUtil、accCardMemUsage一直为-1

9878066f8dc888a28b6066524979ba7

在训练任务的详情页，可以看到卡利用率和卡内存使用率

e09a7454b9268eb15b4351b5fadcb6a

天数的训练任务，监控接口返回的accCardUtil、accCardMemUsage一直为-1 ![9878066f8dc888a28b6066524979ba7](/attachments/e0bc7d7a-34b2-4caa-b249-554ca4ba064a) 在训练任务的详情页，可以看到卡利用率和卡内存使用率 ![e09a7454b9268eb15b4351b5fadcb6a](/attachments/4cc6553a-7c0e-4f02-bf08-dc50ab9e81d1)
9878066f8dc888a28b6066524979ba7.png
61 KiB
e09a7454b9268eb15b4351b5fadcb6a.png
24 KiB
yangxzh1 commented 1 year ago
Collaborator

天数的资源监控数据已经正常了

天数的资源监控数据已经正常了
linfj added this to the v4.3.4 milestone 1 year ago
yangxzh1 added the
测试通过
label 1 year ago
yangxzh1 removed the
开发完成
label 1 year ago
yangxzh1 removed the
测试中
label 1 year ago
yangxzh1 commented 1 year ago
Collaborator

记录一下测试数据：

1、npu

image

2、gcu

命令：efsmi -dmon

image

3、mlu

命令：cnmon -r -t 1000

image

记录一下测试数据： 1、npu ![image](/attachments/a0456b43-2e7e-44d6-981c-de204511aac4) 2、gcu 命令：efsmi -dmon ![image](/attachments/b1729668-87c5-41d3-951c-5f32d78ebc95) 3、mlu 命令：cnmon -r -t 1000 ![image](/attachments/85fbf939-c82e-4c13-a9c5-10ff919e38e2)
image.png
87 KiB
image.png
120 KiB
image.png
104 KiB
image.png
85 KiB
yangxzh1 commented 1 year ago
Collaborator

gpu的测试数据：

任务资源监控页面，英伟达gpu的显存使用率数据跟驱动看到的数据不一样，驱动命令出来的数据，计算4402/12288=36%

image

image

看起来监控页面的数据跟下面这个数据是对的上的。

fa10145bd26cfb58646e1b3789744f9

gpu的测试数据： 任务资源监控页面，英伟达gpu的显存使用率数据跟驱动看到的数据不一样，驱动命令出来的数据，计算4402/12288=36% ![image](/attachments/a9f27bc7-5ba6-41fd-847d-ef00a787dbfd) ![image](/attachments/e4760c07-07f0-47e7-9ef3-f6203543498b) 看起来监控页面的数据跟下面这个数据是对的上的。 ![fa10145bd26cfb58646e1b3789744f9](/attachments/f2f8d185-73e1-4f65-80d9-968ceac2d567)
yangxzh1 closed this issue 11 months ago
Sign in to join this conversation.
No Branch/Tag Specified
Clear branch or tag
Branches Tags
Labels
Apply labels
Clear labels
bug
Something is not working duplicate
This issue or pull request already exists enhancement
New feature help wanted
Need some help invalid
Something is wrong optimizing
Optimizing question
More information is needed wontfix
This won't be fixed 已解决 开发中 开发完成 新建 测试不通过 测试中 测试通过 跟进中
No Label
bug
duplicate
enhancement
help wanted
invalid
optimizing
question
wontfix
已解决
开发中
开发完成
新建
测试不通过
测试中
测试通过
跟进中
Milestone
Set milestone
Clear milestone
No items
No Milestone
v4.3.4
Assignees
Assign users
Clear assignees
No Assignees
 wakinzhang
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes
Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit