[FEAT] support backend and route config through Kubernetes CRDs

[rootfs]

To allow dynamic configuring and updating vLLM backends and routing policies, this PR introduces two additional CRDs: Route and Backend. As illustrated below, Backend abstracts vLLM services, while Route aggregates Backends. The Route controller creates a ConfigMap upon reconciling Route, thus dynamically updating vllm_router.

graph LR
    B[Backend CRD] -->|references| S[Secret]
    R[Route CRD] -->|references| B
    R -->|creates| RC[Route ConfigMap]
    SR[StaticRoute CRD] -->|creates| SRC[StaticRoute ConfigMap]
    
    RC -->|consumed by| Router
    SRC -->|consumed by| Router
    subgraph "Router Controller"
        B
        R
        SR
    end
    subgraph "Generated Resources"
        SRC
    end
    subgraph "vLLM Router"
        Router
    end
    classDef crd fill:#f9f,stroke:#333,stroke-width:2px
    classDef config fill:#bbf,stroke:#333,stroke-width:2px
    classDef component fill:#bfb,stroke:#333,stroke-width:2px
    class B,R,SR crd
    class RC,SRC config
    class Router component

Loading

sequenceDiagram
    participant User
    participant RC as Router Controller
    participant K8s as Kubernetes API
    participant VR as vllm_router
    User->>K8s: Apply Backend CRD
    User->>K8s: Apply Route CRD
    User->>K8s: Apply StaticRoute CRD
    
    RC->>K8s: Watch for CRD changes
    
    K8s-->>RC: Notify CRD creation/update
    
    RC->>K8s: Create/Update ConfigMaps
    Note over RC,K8s: Generates dynamic_config.json
    
    VR->>K8s: Mount ConfigMap
    Note over VR,K8s: Uses --dynamic-config-json flag
    
    VR->>VR: Load configuration
    Note over VR: Apply routing rules<br/>Handle requests based on configuration

Loading