Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix concat spmd rule when input is one dimensional #71253

Conversation

jeff41404
Copy link
Contributor

@jeff41404 jeff41404 commented Feb 24, 2025

PR Category

Auto Parallel

PR Types

Bug fixes

Description

pcard-89497
when use distributed auto parallel, executing concat error on 1D tensors can be reproduced by the following code. this PR will fix this issue. By the way, fix issue of concat error when put_along_axis use concat in dynamic shape.

# python -m paddle.distributed.launch --gpus=0,1,2,3,4,5,6,7 --log_dir logs xxx.py
import paddle
import paddle.distributed as dist

paddle.seed(42)
x = paddle.randn(shape=[16], dtype='float32')
y = paddle.randn(shape=[8], dtype='float32')
z = paddle.concat([x, y]) # ok, shape is [24]

mesh = dist.ProcessMesh([0, 1, 2, 3, 4, 5, 6, 7], dim_names=['dp'])
dist_x = dist.shard_tensor(x, mesh, [dist.Replicate()])
# dist_x = dist.shard_tensor(x, mesh, [dist.Shard(0)]) # shard x/y will get the same result
dist_y = dist.shard_tensor(y, mesh, [dist.Replicate()])
dist_z = paddle.concat([dist_x, dist_y]) # will report error

image

Copy link

paddle-bot bot commented Feb 24, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Copy link
Contributor

@From00 From00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhiqiu zhiqiu merged commit fa16d0d into PaddlePaddle:develop Feb 26, 2025
34 checks passed
@jeff41404 jeff41404 deleted the fix_concat_spmd_rule_when_input_is_one_dimensional branch February 27, 2025 02:02
Enigmatisms pushed a commit to Enigmatisms/Paddle that referenced this pull request Mar 6, 2025
* fix concat spmd rule when input is one dimensional

* add unit test case

* fix issue of coverage and add unit test case

---------

Co-authored-by: Glencsa <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants