Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Fix PIR Unittest No.550,555】Fix some test case in PIR #65602

Merged
merged 1 commit into from
Jul 2, 2024

Conversation

wanghuancoder
Copy link
Contributor

@wanghuancoder wanghuancoder commented Jul 1, 2024

PR Category

Others

PR Types

Others

Description

修复rnn下test_rnn_nets_static.py单测。
rnn使用了老静态图的create_variable机制。用于支持rnn中state的生成:
27A2D0E5E0F62003DA13011258C2DB39
根据分析,其特点是:

  1. 初始值必须没有显存,处于uninitialized状态,因为第一次,和剩下的时候,用的cudnn API不一样
  2. persistable必须是True,否则执行器会GC显存,无法给下一轮保存数据
  3. 不能参与优化器的参数优化
  4. 不需要save,因为test时不需要这个值

PIR使用

                self._dropout_state = paddle.pir.core.create_parameter(
                    dtype="uint8",
                    shape=[0],
                    name=f"dropout_state{NON_PERSISTABLE_VAR_NAME_SUFFIX}",
                    initializer=paddle.nn.initializer.Constant(value=0),
                    trainable=False,
                )
                self._dropout_state.stop_gradient = True
  1. shape=[0]初始化的Tensor为uninitialized
  2. create_parameter的Value,persistable为True
  3. stop_gradient = True,可以不需要优化
  4. NON_PERSISTABLE_VAR_NAME_SUFFIX在save里有特殊处理,不会save

关联Issue:#63740

Pcard-67164

Copy link

paddle-bot bot commented Jul 1, 2024

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@wanghuancoder wanghuancoder merged commit 99bbfe6 into PaddlePaddle:develop Jul 2, 2024
31 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants