Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Audio] Support Whisper V3 #1147

Merged
merged 3 commits into from
Feb 14, 2025
Merged

[Audio] Support Whisper V3 #1147

merged 3 commits into from
Feb 14, 2025

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented Feb 13, 2025

Purpose

  • Support Whisper V3 model

Changes

  • Change default whisper model to v3
  • Modify preprocessing function to be simpler
  • Add dtype conversion to preprocessing function
    • Note that this is only required for feature extractor processors, as they return values which are float types (not just token ids, which work regardless of model dtype)

Follow-ups

  • Dtype conversion should theoretically be injected into prebaked dataset pathways as well, although I consider this low priority since we push users towards writing their own data processing functions

Testing

  • Quantized Whisper v3 model
  • Note that you may have to add ds.cleanup_cache_files() to line 40 in order to overwrite any existing mapping caches

Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

Copy link
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@kylesayrs kylesayrs added the ready When a PR is ready for review label Feb 14, 2025
@kylesayrs kylesayrs self-assigned this Feb 14, 2025
@dsikka dsikka enabled auto-merge (squash) February 14, 2025 20:21
@dsikka dsikka merged commit c8091d3 into main Feb 14, 2025
7 checks passed
@dsikka dsikka deleted the kylesayrs/support-whisperv3 branch February 14, 2025 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants