Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Mocked vLLM App with Dynamic Metrics for Autoscaling Efficiency #117

Closed
kr11 opened this issue Sep 2, 2024 · 0 comments · Fixed by #116
Closed

Enhance Mocked vLLM App with Dynamic Metrics for Autoscaling Efficiency #117

kr11 opened this issue Sep 2, 2024 · 0 comments · Fixed by #116
Assignees
Labels
area/autoscaling kind/feature Categorizes issue or PR as related to a new feature.

Comments

@kr11
Copy link
Collaborator

kr11 commented Sep 2, 2024

🚀 Feature Description and Motivation

I propose adding a 'metrics' endpoint in docs/development/app/app.py that returns Prometheus-style metrics results.

By mocking the impact of replica changes on metrics, where app.py will detect the replica number and return inversely proportional metric values, we can improve the testing and development of autoscaling policies.

We further add an unitest and enrich README.

Use Case

No response

Proposed Solution

related PR: #116

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/autoscaling kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
1 participant