[Feat] Integrate deployment configurations and fix autoscaler/gpu optimizer connectivity

[zhangjyr]

Pull Request Description

This PR fixed the connectivity problem between podautoscaler and GPU optimizer by:

1. Updated k8s role definition to use ClusterRole, so GPU optimizer now monitor all deployments in all namespace with model label
2. Include [WIP] Add GPU Optimizer deployment and update configurations #480 changes. Deployment configurations are integrated into config/default

Note: I keep the deployment.yaml under GPU optimizer undeleted for debugging purposes.

Related Issues

Resolves: #484 #480 #459

Important: Before submitting, please complete the description above and review the checklist below.

---

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

• [Bug]: Corrections to existing functionality
• [CI]: Changes to build process or CI pipeline
• [Docs]: Updates or additions to documentation
• [API]: Modifications to aibrix's API or interface
• [CLI]: Changes or additions to the Command Line Interface
• [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

• PR title includes appropriate prefix(es)
• Changes are clearly explained in the PR description
• New and existing tests pass successfully
• Code adheres to project style and best practices
• Documentation updated to reflect changes (if applicable)
• Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

[zhangjyr]

I moved some comments from #480 here:
If the server is down, what's the autoscaler behavior? Have you tested such behaviors?
We haven't tested. Ideally, if the GPU optimizer were down, the podautoscaler would not be able to read metrics and keep replicas intact. After the GPU optimizer resumes, the GPU optimizer should not output valid metrics before a new solution is reached with sufficient load traces. Most of the changes involved in this approach will be easy to implement. However, we'll need to prolong the load trace timeout in Redis (e.g., up to 300 seconds, which is aligned with the current GPU optimizer window), so the GPU optimizer can restore the solution quickly enough.

[nwangfw]

Right now, we are modifying code under python/aibrix/aibrix/gpu_optimizer. Can we delete python/aibrix/aibrix/gpuoptimizer folder?

[Jeffwan]

@nwangfw we should not have such folder? upstream folder has been renamed

[nwangfw]

@nwangfw we should not have such folder? upstream folder has been renamed

Never mind. Seems that it has been deleted. Please ignore it and thanks.

[Jeffwan]

/lgtm