[Distributed] [CustomDevices] Adapt SP on lora && polish MC2 APIs

[SylarTiaNII]

PR types

New features & Performance optimization

PR changes

Others

Description

• Adapt sequence parallel on lora (gain 14%+ performance on llama-13b && save lots of memory on long sequence scenes) (cp from [Distributed] adapt sequence parallel on LoRA #8235)
• polish MC2 APIs for better structure

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 17.18213% with 241 lines in your changes are missing coverage. Please review.

Project coverage is 55.25%. Comparing base (4039897) to head (4a0982e).
Report is 2 commits behind head on develop.

Files	Patch %	Lines
paddlenlp/peft/lora/lora_layers.py	14.49%	118 Missing ⚠️
paddlenlp/transformers/mc2_parallel_linear.py	18.46%	106 Missing ⚠️
paddlenlp/peft/lora/lora_model.py	25.00%	15 Missing ⚠️
paddlenlp/transformers/llama/modeling.py	33.33%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8303      +/-   ##
===========================================
- Coverage    55.33%   55.25%   -0.08%     
===========================================
  Files          614      613       -1     
  Lines        95341    95625     +284     
===========================================
+ Hits         52753    52837      +84     
- Misses       42588    42788     +200     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.