[Prediction] Update LLM prediction. #8778
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #8778 +/- ##
===========================================
+ Coverage 55.03% 55.53% +0.50%
===========================================
Files 627 631 +4
Lines 98921 98466 -455
===========================================
+ Hits 54440 54685 +245
+ Misses 44481 43781 -700 ☔ View full report in Codecov by Sentry. |
DesmonDay
force-pushed
the
update_predict_new
branch
from
a19f4b6 to
6944c25
Compare
DesmonDay
force-pushed
the
update_predict_new
branch
from
6944c25 to
33fde67
Compare
ZHUI
reviewed
Jul 19, 2024
| # TODO: Support safetensors loading. | ||
| kwargs["use_safetensors"] = False | ||
| return super().from_pretrained(pretrained_model_name_or_path, *args, **kwargs) | ||
| config = kwargs.pop("config", None) |
DesmonDay
force-pushed
the
update_predict_new
branch
from
0d1026f to
ea2f41c
Compare
DesmonDay
force-pushed
the
update_predict_new
branch
from
ea2f41c to
5c2f595
Compare
…nto update_predict_new
DesmonDay
force-pushed
the
update_predict_new
branch
from
99c031b to
f1d65ab
Compare
ZHUI
previously approved these changes
Jul 23, 2024
| from safetensors.numpy import load_file as safe_load_file | ||
|
|
||
|
|
||
| def load_sharded_checkpoint(folder, variant=None, return_numpy=False): |
There was a problem hiding this comment.
是不是可以直接放到paddlenlp/transformers/model_utils.py里作为类函数？所有实现的类都可以直接调用
DesmonDay
force-pushed
the
update_predict_new
branch
from
b7f32af to
e2cc486
Compare
ZHUI
reviewed
Jul 25, 2024
| from safetensors.numpy import save_file as safe_save_file | ||
|
|
||
| if sys.platform.startswith("win"): |
DesmonDay
force-pushed
the
update_predict_new
branch
from
e2cc486 to
ebc553d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Function optimization
PR changes
Others
Description
Update LLM prediction, support safetensors model download and loading.