#632 master-fix-optimizer

Merged
zoulq merged 8 commits from frelam/MSAdapter:master-fix-optimizer into master 9 months ago
frelam reviewed 9 months ago
@@ -163,4 +172,2 @@
for name in ms_params_name:
_param_list = []
ms_params = getattr(self, name)
for p in ms_params:
frelam commented 9 months ago
之前只考虑了ParameterTuple的形式 没有考虑单个Parameter
frelam reviewed 9 months ago
msadapter/pytorch/optim/optimizer.py
@@ -173,0 +183,4 @@
_load(ms_params[i], state_tensor[i])
else:
_data = cast_to_ms_tensor(state_tensor)
ms_params.set_data(_data)
frelam commented 9 months ago
使用Parameter.set_data的形式, 不适用assign 进入到_ms_load_state_dict这个函数里面的, 都是为了保存Parameter, 其他的不进入该函数。
frelam commented 9 months ago
Poster
修复Optimizer._ms_load_state_dict/_ms_state_dict 中, 只支持ParameterTuple, 不支持Parameter 的bug 最新的mindspore优化器, 存在Parameter的情况
zoulq reviewed 9 months ago
@@ -163,0 +167,4 @@
for p in ms_params:
_state.append(_save(p))
else:
_state = tensor(ms_params.asnumpy())
zoulq commented 9 months ago
这里为什么要先asnumpy再tensor初始化,直接获取params.data呢?
frelam commented 9 months ago
params是mindspore的Parameters, params.data 是用return self实现的, 还是返回Parameter 其实不asnumpy流程也是能够走通的, 主要是tensor(Parameter)用法有点奇怪而已。
zoulq reviewed 9 months ago
@@ -173,0 +182,4 @@
if isinstance(ms_params, Iterable):
if not isinstance(state_tensor, Iterable) or len(ms_params) != len(state_tensor):
raise ValueError(f"state_dict of ms_param '{name}' is not correct. please check. "
f"(ms_param '{name}' is Iterable, but state_dict['{name}'] is not.)")
zoulq commented 9 months ago
还有len不相等的情况,报错内容更新一下
frelam commented 9 months ago
ok。 已经分开判断这两个条件, 分别报错。
zoulq merged commit ea4fc66b40 into master 9 months ago
The pull request has been merged as ea4fc66b40.
Sign in to join this conversation.
No reviewers
No Label
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This pull request currently doesn't have any dependencies.

Loading…
There is no content yet.