You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
to discover all the models deployed in the AIBrixsystem, it will be useful to have a single endpoint which displays all the models like the openai /models endpoint
Use Case
to discover all the models deployed in the AIBrixsystem, it will be useful to have a single endpoint which displays all the models like the openai /models endpoint
Proposed Solution
No response
The text was updated successfully, but these errors were encountered:
We get few users report this issue. This would be implemented shortly. We talked about two different options
AIBrix should provides an abstraction to provide all available models. the challenge is current plugin design does not like to manipulate the response. Technically, it should aggregate the models (including lora adapter) from the cache. data plane traffic works out-of-box in multi-model scenarios.
refactor plugin to provide model scope /v1/models return results. That means it only care about single base + all the related adapters. but seems this way need more efforts and can not meet production users's need.
Currently, I prefer the 1st way and we need to do such trick for such api. It brings another problem on the api compatibility and we should revisit it now. #732
🚀 Feature Description and Motivation
to discover all the models deployed in the AIBrixsystem, it will be useful to have a single endpoint which displays all the models like the openai /models endpoint
Use Case
to discover all the models deployed in the AIBrixsystem, it will be useful to have a single endpoint which displays all the models like the openai /models endpoint
Proposed Solution
No response
The text was updated successfully, but these errors were encountered: