#921 update some api

Merged
Erpim merged 12 commits from frelam/MSAdapter:master0419 into master 1 week ago
frelam commented 1 week ago
frelam changed title from update some api to [WIP]update some api 1 week ago
frelam reviewed 1 week ago
@@ -0,0 +66,4 @@

self.exp_avg = cast_to_float32(self.exp_avg)
self.exp_avg_sq = cast_to_float32(self.exp_avg_sq)
self.max_exp_avg_sq = cast_to_float32(self.max_exp_avg_sq)
frelam commented 1 week ago
pytorch能够在第一个step之后, 自动将该变量从float16变成float32. 但mindspore的assign函数, 不支持float32 assign 给一个float16 变量。 所以这里提前做好cast
zoulq commented 1 week ago
pytorch是如何实现float32转float16类型?
frelam commented 1 week ago
float16转float32是吗? pytorch计算的结果是float32, 然后直接把原来的float16 tensor丢掉,换上新计算出来的float32 tensor。 我们计算结果也是float32, 但是我们是用parameter tuple中的parameter存的, 不能扔掉, 只能assign更新值。
frelam reviewed 1 week ago
@@ -0,0 +107,4 @@
if grads is None:
grads = []
for param_group in self.param_groups:
for param in param_group['params']:
frelam commented 1 week ago
支持动态替换param_group['params'] 每次都从这里面重新取出新的param
zoulq reviewed 1 week ago
mindtorch/torch/functional.py
@@ -453,2 +456,3 @@

output = ms.ops.linspace(start, end, steps)
# TODO: ms.ops.linspace has some bug now.
# output = ms.ops.linspace(start, end, steps)
zoulq commented 1 week ago
全平台的算子目前实现都有问题吗?
frelam commented 1 week ago
只有ascend有问题, 已加平台判断。
zoulq reviewed 1 week ago
@@ -0,0 +8,4 @@

op_mul = P.Mul()
op_pow = P.Pow()
op_sqrt = P.Sqrt()
zoulq commented 1 week ago
直接用ms.ops.mul/pow/sqrt有什么区别?
frelam commented 1 week ago
没区别我理解。 只是这里更直接一些, 应该可以减少一些python层的消耗。
frelam changed title from [WIP]update some api to update some api 1 week ago
Erpim reviewed 1 week ago
mindtorch/torch/functional.py
@@ -458,1 +461,3 @@
output = ms.ops.linspace(start, end, steps)
# TODO: ms.ops.linspace has some bug on Ascend now.
if is_under_ascend_context():
num = int(steps)
Erpim commented 1 week ago
ms.numpy.linspace是否可用?或者转cpu异构执行?当前实现性能较差
frelam commented 1 week ago
用了, 转cpu异构执行 , 已修改。
Erpim reviewed 1 week ago
@@ -1250,6 +1250,9 @@ class Tensor(StubTensor, metaclass=_TensorMeta):
if dtype == mindtorch_dtype.bfloat16:
return tensor(out.astype(np.float32), dtype=dtype).reshape(target_shape)
return tensor(out).reshape(target_shape)
if isinstance(shape[0], (tuple, list)):
Erpim commented 1 week ago
影响主流场景性能,需评估
frelam commented 1 week ago
单机笔记本测试了 100000次view, 修改前修改后性能上没有明显区别, 完成时间都是8秒多。
Erpim reviewed 1 week ago
@@ -45,2 +46,4 @@
)
return run_function(end + 1, len(functions) - 1, functions)(input)

def detach_variable(inputs):
Erpim commented 1 week ago
新微分方案是否支持?
frelam commented 1 week ago
用到了detach 和 requires_grad, 新微分方案应该是支持的
Erpim reviewed 1 week ago
mindtorch/torch/utils/checkpoint.py
@@ -45,2 +46,4 @@
)
return run_function(end + 1, len(functions) - 1, functions)(input)

def detach_variable(inputs):
Erpim commented 1 week ago
新微分方案是否支持?
frelam marked this conversation as resolved
Erpim reviewed 1 week ago
mindtorch/torch/utils/checkpoint.py
@@ -45,2 +46,4 @@
)
return run_function(end + 1, len(functions) - 1, functions)(input)

def detach_variable(inputs):
Erpim commented 1 week ago
新微分方案是否支持?
frelam marked this conversation as resolved
Erpim merged commit 47f6cd8cce into master 1 week ago
The pull request has been merged as 47f6cd8cce.
Sign in to join this conversation.
No reviewers
No Label
No Milestone
No Assignees
3 Participants
Notifications
Due Date

No due date set.

Dependencies

This pull request currently doesn't have any dependencies.

Loading…
There is no content yet.