Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[VLM] Multimodal Data Collator #1087

Merged
merged 2 commits into from
Jan 23, 2025
Merged

[VLM] Multimodal Data Collator #1087

merged 2 commits into from
Jan 23, 2025

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented Jan 20, 2025 •
edited
Loading

Purpose

Changes

  • Remove data collator definitions in LC
  • Add data collators in examples with a comment indicating that this is for multimodal inputs

Testing

Ran all multimodal vision models

  • Qwen2
  • Pixtral
  • Mllama
  • Llava
  • Phi3_vision

@kylesayrs kylesayrs mentioned this pull request Jan 20, 2025
Merged
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@kylesayrs kylesayrs added the ready When a PR is ready for review label Jan 21, 2025
@kylesayrs kylesayrs marked this pull request as ready for review January 21, 2025 04:52
@kylesayrs kylesayrs requested a review from mgoin January 22, 2025 18:19
dsikka
dsikka approved these changes Jan 23, 2025
View reviewed changes
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit about removing empty file otherwise LGTM

@dsikka dsikka merged commit 6377f1e into main Jan 23, 2025
7 of 8 checks passed
@dsikka dsikka deleted the kylesayrs/generalized_data_collator branch January 23, 2025 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dsikka dsikka dsikka approved these changes

@horheynm horheynm horheynm approved these changes

@mgoin mgoin Awaiting requested review from mgoin

Assignees
No one assigned
Labels
ready When a PR is ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kylesayrs @dsikka @horheynm