Feat: Router observability (Current QPS, router-side queueing delay, etc) Part 1#119

Merged

YuhanLiu11 merged 15 commits intovllm-project:mainfrom sitloboi2012:feat/update-prom-dashboard

Feb 19, 2025

+1,057-598

Commits on Feb 18, 2025

update vllm-dashboard and router to contain add on metrics such as coree vLLM metrics, operational metrics, router observe metrics, update requirement.txt for router and tests, update install-minik…
sitloboi2012
committed
update the router and vllm-dashboard to align with the reference from @YuhanLiu11
sitloboi2012
committed
update the run router for case with static service discovery and k8s service discovery
sitloboi2012
committed
rebase with main
sitloboi2012
committed
update vllm-dashboard and router to contain add on metrics such as coree vLLM metrics, operational metrics, router observe metrics, update requirement.txt for router and tests, update install-minik…
sitloboi2012
committed
update the router and vllm-dashboard to align with the reference from @YuhanLiu11
sitloboi2012
committed
run pre-commit for linting
sitloboi2012
committed
remove conflict info data in router.py
sitloboi2012
committed
running pre-commit to make sure linting pass
sitloboi2012
committed
update vllm dashboard to remove redundancy
sitloboi2012
committed
remove redundancy in run-router.sh
sitloboi2012
committed
update the average itl, and other metrics into router.py, engine_stats.py and request_stats.py, update observe/install.sh to helm update to solve the problem already run helm service same name, upd…
sitloboi2012
committed
update based on Shaoting-Feng comment
sitloboi2012
committed
modify to keep the original comment info
sitloboi2012
committed
run pre-commit for formatting
sitloboi2012
committed