You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I understand that if i use the router in a kubernetes environment, the router uses service discovery to build a pool of backend vllm services. While this worry-free mechanism is good in many ways, i am wondering if we can add yet another way to explicitly declare a backend connectivity through a kubernetes crd. The idea is that, for certain use cases, the crd represents a persistent configuration that can be managed by controllers (while annotation and labels that are fed to service discovery are mostly ad hoc)
Why do you need this feature?
Support routing CRD so that a declarative management can be supported instead of using service discovery.
Describe the feature
I understand that if i use the router in a kubernetes environment, the router uses service discovery to build a pool of backend vllm services. While this worry-free mechanism is good in many ways, i am wondering if we can add yet another way to explicitly declare a backend connectivity through a kubernetes crd. The idea is that, for certain use cases, the crd represents a persistent configuration that can be managed by controllers (while annotation and labels that are fed to service discovery are mostly ad hoc)
Why do you need this feature?
Support routing CRD so that a declarative management can be supported instead of using service discovery.
Additional context
(see the slack discussion https://vllm-dev.slack.com/archives/C089SMEAKRA/p1740756084674449)
The text was updated successfully, but these errors were encountered: