[AutoParallel] support GPT for auto_parallel

[liym27]

PR types

New features

PR changes

Models

Description

support GPT for auto_parallel

1. 在GPT上验证自动并行架构

• 对比动态图半自动并行和动态图手动并行，收敛和精度结果符合预期，涉及以下策略验证
  • mp2
  • dp2mp2
  • dp2mp2pp2
  • dp2mp2pp2 + 动转静
  • dp2mp2pp2 + 动转静 + amp

  dp 数据并行，mp 模型并行，pp 流水线并行， amp 自动混合精度

2. 本 PR 代码包含最终模型代码、示例脚本、单测。验证结果如下

mp2

dp2mp2

dp2mp2pp2

dp2mp2pp2 + 动转静

dp2mp2pp2 + 动转静 + amp

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 16.50672% with 435 lines in your changes are missing coverage. Please review.

Project coverage is 54.88%. Comparing base (7b493a8) to head (f368f57).
Report is 2 commits behind head on develop.

❗ Current head f368f57 differs from pull request most recent head 1b0f2ef. Consider uploading reports for the commit 1b0f2ef to get more accurate results

Files	Patch %	Lines
paddlenlp/transformers/gpt/modeling_auto.py	16.34%	435 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8160      +/-   ##
===========================================
- Coverage    55.15%   54.88%   -0.27%     
===========================================
  Files          601      603       +2     
  Lines        91764    92371     +607     
===========================================
+ Hits         50611    50699      +88     
- Misses       41153    41672     +519     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM