Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metric standardizing in ai runtime #163

Merged
merged 8 commits into from
Sep 13, 2024
Merged

Conversation

brosoul
Copy link
Collaborator

@brosoul brosoul commented Sep 11, 2024

Pull Request Description

AI Runtime merge self metrics with enigne metrics. At present, only Rename standardizing rule of the metrics of the main pod is supported. The main data structures added are as follows:

  • HTTPCollector provides metrics collect from other http source
  • StandardRule provides the detail implemention of metric merge
  • XXX_METRIC_STANDARD_RULES in engine_rules.py records metric merge rules for various inference engines

As shown in the following figure, AI Runtime rename vllm:request_success_total to aibrix:request_success_total according to the rules VLLM_METRIC_STANDARD_RULES (Meanwhile, the original indicators were retained).
image

Related Issues

Resolves: #106

Important: Before submitting, please complete the description above and review the checklist below.


Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

  • [Bug]: Corrections to existing functionality
  • [CI]: Changes to build process or CI pipeline
  • [Docs]: Updates or additions to documentation
  • [API]: Modifications to aibrix's API or interface
  • [CLI]: Changes or additions to the Command Line Interface
  • [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

  • PR title includes appropriate prefix(es)
  • Changes are clearly explained in the PR description
  • New and existing tests pass successfully
  • Code adheres to project style and best practices
  • Documentation updated to reflect changes (if applicable)
  • Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

@brosoul brosoul changed the title [WIP] metric standardizing Metric standardizing Sep 12, 2024
@brosoul brosoul requested a review from Jeffwan September 12, 2024 07:56
@Jeffwan Jeffwan changed the title Metric standardizing Metric standardizing in ai runtime Sep 12, 2024
@Jeffwan
Copy link
Collaborator

Jeffwan commented Sep 12, 2024

BTW, beside the filtered metrics, do you think we should expose rest of the application metrics? Let's have say

aibrix:request_total
vllm:some_other_uncommon_metrics

@brosoul
Copy link
Collaborator Author

brosoul commented Sep 13, 2024

BTW, beside the filtered metrics, do you think we should expose rest of the application metrics? Let's have say顺便说一句,除了过滤的指标之外,您认为我们应该公开其余的应用程序指标吗?让我们说

aibrix:request_total
vllm:some_other_uncommon_metrics

Currently, my approach is to retain the original engine metrics while adding some new metrics based on the rules.

@Jeffwan
Copy link
Collaborator

Jeffwan commented Sep 13, 2024

the change looks good to me

@Jeffwan Jeffwan merged commit c9e1972 into main Sep 13, 2024
3 checks passed
@Jeffwan Jeffwan deleted the linhui/metric-standardizing branch September 13, 2024 03:23
gangmuk pushed a commit that referenced this pull request Jan 25, 2025
* feat: init standard metric framework

* feat: add engine metric scrape support

* format

* format

* adjust server run port

* mv constant config to config.py

* style

* fix: vllm metric standard rule
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AI Runtime merge self metrics with enigne metrics
2 participants