Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon 8th No.36】Add gme-Qwen2-VL for PaddleMix #1103

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

ZhijunLStudio
Copy link
Contributor

@ZhijunLStudio ZhijunLStudio commented Mar 10, 2025

Add gme-Qwen2-VL for PaddleMix

[New Feature] Implement GmeQwen2VL Model and Multimodal Inference Pipeline

Changes Proposed

  1. Model Architecture & Inference Pipeline Implementation

    • Added the text, image, and fused embedding computation modules of the GmeQwen2VL model to paddlemix/models.
    • Integrated a multimodal retrieval task example in paddlemix/examples, including text-to-image similarity calculation and information retrieval workflows.
  2. Functional Enhancements

    • Supports custom instructions for text embeddings (e.g., "Find an image that matches the given text.").
    • Provides fused embedding interfaces (get_fused_embeddings()) and encoding functions for queries/databases (encode_queries() / encode_corpus()).
  3. Performance Validation

    • Achieved performance comparable to the original repository through internal testing, with validated accuracy in similarity calculations (see documentation for test results).

Copy link

paddle-bot bot commented Mar 10, 2025

Thanks for your contribution!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

把这些图片去掉吧;用到的地方使用链接的方式(把图片拉到issue里,会生成相关的链接)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

py文件都加上copyright吧;可以从已有的文件里copy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

lyuwenyu
lyuwenyu previously approved these changes Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants