Skip to content

【Fix PIR Unittest No.550,555】Fix some test case in PIR - #65602

Merged
wanghuancoder merged 1 commit into
PaddlePaddle:developfrom
wanghuancoder:fix_pir_rnn5
Jul 2, 2024
Merged

wanghuancoder merged 1 commit into
PaddlePaddle:developfrom
wanghuancoder:fix_pir_rnn5

Conversation

@wanghuancoder

@wanghuancoder wanghuancoder commented Jul 1, 2024 •

Copy link
Copy Markdown
Contributor

PR Category

Others

PR Types

Others

Description

修复rnn下test_rnn_nets_static.py单测。
rnn使用了老静态图的create_variable机制。用于支持rnn中state的生成:
27A2D0E5E0F62003DA13011258C2DB39
根据分析,其特点是:

  1. 初始值必须没有显存,处于uninitialized状态,因为第一次,和剩下的时候,用的cudnn API不一样
  2. persistable必须是True,否则执行器会GC显存,无法给下一轮保存数据
  3. 不能参与优化器的参数优化
  4. 不需要save,因为test时不需要这个值

PIR使用

                self._dropout_state = paddle.pir.core.create_parameter(
                    dtype="uint8",
                    shape=[0],
                    name=f"dropout_state{NON_PERSISTABLE_VAR_NAME_SUFFIX}",
                    initializer=paddle.nn.initializer.Constant(value=0),
                    trainable=False,
                )
                self._dropout_state.stop_gradient = True
  1. shape=[0]初始化的Tensor为uninitialized
  2. create_parameter的Value,persistable为True
  3. stop_gradient = True,可以不需要优化
  4. NON_PERSISTABLE_VAR_NAME_SUFFIX在save里有特殊处理,不会save

关联Issue:#63740

Pcard-67164

@paddle-bot

paddle-bot Bot commented Jul 1, 2024

Copy link
Copy Markdown

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@wanghuancoder
wanghuancoder merged commit 99bbfe6 into PaddlePaddle:develop Jul 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants