Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix FusedLinearWithGradAdd bug #8178

Merged
merged 1 commit into from
Mar 25, 2024

Conversation

MarioLulab
Copy link
Contributor

@MarioLulab MarioLulab commented Mar 25, 2024

PR types

Bug fixes

PR changes

Others

Description

llm/gpt3 模型在开启 enable_linear_fused_grad_add 策略后会报错:
image

具体定位到问题出现在 FusedLinearWithGradAdd 的 backward 方法中。FusedLinearWithGradAdd.backward 会调用 fused_linear_param_grad_add 算子进行计算,而 _C_ops.fused_linear_param_grad_add api 接收六个参数输入:

(Tensor x, Tensor dout, Tensor dweight, Tensor dbias, bool multi_precision = true, bool has_bias = true)

当前 FusedLinearWithGradAdd 在调用 _C_ops.fused_linear_param_grad_add 时缺少第六个参数(has_bias)参数的输入,导致 C++ api 没法正确解析出 has_bias 参数,进而报错

补充说明:
#6602 也修复了类似的问题

Copy link

paddle-bot bot commented Mar 25, 2024

Thanks for your contribution!

Copy link
Collaborator

@ZHUI ZHUI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

codecov bot commented Mar 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 55.41%. Comparing base (db49062) to head (5742067).
Report is 54 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8178      +/-   ##
===========================================
- Coverage    56.56%   55.41%   -1.16%     
===========================================
  Files          589      597       +8     
  Lines        89964    91594    +1630     
===========================================
- Hits         50889    50754     -135     
- Misses       39075    40840    +1765     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ZHUI ZHUI merged commit a5d87f5 into PaddlePaddle:develop Mar 25, 2024
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants