代码拉取完成，页面将自动刷新
| name | about | labels |
|---|---|---|
| Bug Report | Use this template for reporting a bug | kind/bug |
Ascend/GPU/CPU):Uncomment only one
/device <>line, hit enter to put that in a new line, and remove leading whitespaces from that line:/device gpu
test_ms_pinns_navier_stokes_pynative_train_check_fps.py
网络使用pynative模式运行失败
网络训练成功
Traceback (most recent call last):
File "train.py", line 52, in <module>
train_navier(**conf)
File "/data/zjc/workspace/solution_test/remaining/test_scripts/mindspore/net/pinns/pinns_navier_stokes_pynative/src/NavierStokes/train_ns.py", line 108, in train_navier
callbacks=[LossMonitor(loss_print_num), ckpoint, TimeMonitor(1), eval_cb], dataset_sink_mode=True)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 718, in train
sink_size=sink_size)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 502, in _train
self._train_dataset_sink_process(epoch, train_dataset, list_callback, cb_params, sink_size)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 564, in _train_dataset_sink_process
outputs = self._train_network(*inputs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 433, in __call__
raise err
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 430, in __call__
output = self.run_construct(cast_inputs, kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 352, in run_construct
output = self.construct(*cast_inputs, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/dataset_helper.py", line 79, in construct
return self.network(*outputs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 433, in __call__
raise err
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 430, in __call__
output = self.run_construct(cast_inputs, kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 352, in run_construct
output = self.construct(*cast_inputs, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/wrap/cell_wrapper.py", line 353, in construct
loss = self.network(*inputs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 433, in __call__
raise err
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 430, in __call__
output = self.run_construct(cast_inputs, kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 352, in run_construct
output = self.construct(*cast_inputs, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/wrap/cell_wrapper.py", line 110, in construct
out = self._backbone(data)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 433, in __call__
raise err
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 430, in __call__
output = self.run_construct(cast_inputs, kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 352, in run_construct
output = self.construct(*cast_inputs, **kwargs)
File "/data/zjc/workspace/solution_test/remaining/test_scripts/mindspore/net/pinns/pinns_navier_stokes_pynative/src/NavierStokes/net.py", line 224, in construct
d_p = self.dp(x, y, t)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 433, in __call__
raise err
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 430, in __call__
output = self.run_construct(cast_inputs, kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 352, in run_construct
output = self.construct(*cast_inputs, **kwargs)
File "/data/zjc/workspace/solution_test/remaining/test_scripts/mindspore/net/pinns/pinns_navier_stokes_pynative/src/NavierStokes/net.py", line 138, in construct
return self.grad(self.net)(x, y, t, (sens_1, sens_2))
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/common/api.py", line 77, in wrapper
results = fn(*arg, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/ops/composite/base.py", line 376, in after_grad
self._pynative_forward_run(grad_, args, kwargs, fn)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/ops/composite/base.py", line 355, in _pynative_forward_run
fn(*args, **new_kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 440, in __call__
_pynative_executor.end_graph(self, output, *inputs, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/common/api.py", line 376, in end_graph
self._executor.end_graph(obj, output, *args, *(kwargs.values()))
RuntimeError: _Map_base::at
pinns网络使用pynative模式在gpu环境运行失败
二阶在缓存下逻辑还有问题，同一个网络先跑一阶，在跑二阶。前面一阶的缓存会影响二阶的逻辑。
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
#Appearance & Root Cause
pynative三阶以上的缓存存在问题。第2个step时候，三阶取到了2阶的缓存，导致运行错乱。
#Fix Solution
对每一阶缓存添加特别标识，让其准确取值。
Relation PR
https://e.gitee.com/mind_spore/repos/mindspore/mindspore/pulls/26072
回归版本：2021-11-8 每日构建版本
编译时间 2021-11-8
回归步骤：参考issue复现步骤
基本功能：问题已解决
测试结论：回归通过
回归人员：zhongjicheng
回归时间：2021-11-11
登录 后才可以发表评论
