Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Support Lora management in kubernetes native way #22

Closed
5 of 8 tasks
Jeffwan opened this issue Jul 13, 2024 · 1 comment
Closed
5 of 8 tasks

[RFC] Support Lora management in kubernetes native way #22

Jeffwan opened this issue Jul 13, 2024 · 1 comment
Assignees
Labels
area/lora kind/enhancement New feature or request priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Milestone

Comments

@Jeffwan
Copy link
Collaborator

Jeffwan commented Jul 13, 2024

After the interview discussion, we determine to create custom resource for Lora Adapters. We need to create the api and controllers for it. Detail design will come soon

  • Create the API and controller skeleton Create Lora api and controller #23
  • Design the Lora API
  • Implement the Lora controllers Part I (Service & Endpoint slice controller)
  • Implement the Lora controllers Part II (Interact with inference engine for loading)
  • Finish the routing algorithm design and implementation
  • Design and implement cold start manager
  • Design and implement the lora scheduler
  • Engine related changes Improve Lora management quality to production grade #14
@Jeffwan Jeffwan self-assigned this Jul 13, 2024
@Jeffwan Jeffwan added kind/enhancement New feature or request area/lora labels Jul 13, 2024
@Jeffwan Jeffwan added this to the v0.1.0-rc.0 milestone Jul 13, 2024
@Jeffwan Jeffwan changed the title Implement Lora controllers [RFC] Support Lora management in kubernetes native way Jul 13, 2024
@Jeffwan Jeffwan modified the milestones: v0.1.0-rc.0, v0.1.0 Jul 29, 2024
@Jeffwan Jeffwan added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label Jul 29, 2024
@Jeffwan
Copy link
Collaborator Author

Jeffwan commented Oct 17, 2024

We leave some advanced features like cold start manager and lora scheduler to v0.2.0. We have enough feature coverage in v0.1.0.
#303 #304 #305 have been created to track unfinished items

@Jeffwan Jeffwan closed this as completed Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/lora kind/enhancement New feature or request priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Development

No branches or pull requests

1 participant