Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some metric shows 0 #1478

Open
xianjingfeng opened this issue Jan 24, 2024 Discussed in #1268 · 7 comments
Open

Some metric shows 0 #1478

xianjingfeng opened this issue Jan 24, 2024 Discussed in #1268 · 7 comments

Comments

@xianjingfeng
Copy link
Member

Discussed in #1268

Originally posted by youjeongsue October 27, 2023
Hi. I'm setting up a prometheus pushgateway to collect metrics.

https://github.com/apache/incubator-uniffle/blob/master/docs/metrics_guide.md#report-metrics-to-prometheus-automatically

rss.metrics.prometheus.pushgateway.addr rss-pushgateway...com:80
rss.metrics.prometheus.pushgateway.jobname rss-shuffle-server
rss.metrics.reporter.class org.apache.uniffle.common.metrics.prometheus.PrometheusPushGatewayMetricReporter

However, I only get the JVM related metrics to show up properly, all the others are collected as zeros.
image
image
image

If I'm setting this up wrong, please correct me. (__)
Your advice will be of great help to me.

@lifeSo
Copy link
Collaborator

lifeSo commented Jan 26, 2024

I have not see it , and here is our conf and it it very simple:

rss.metrics.reporter.class org.apache.uniffle.common.metrics.prometheus.PrometheusPushGatewayMetricReporter
rss.metrics.prometheus.pushgateway.addr   <normal_ip>:9091
**rss.metrics.prometheus.pushgateway.report.interval.seconds 20**
rss.metrics.prometheus.pushgateway.jobname  <normal_jobname>

How about add this:
rss.metrics.prometheus.pushgateway.report.interval.seconds

@BaluDontu
Copy link

Hi,

I am seeing a similar issue where I see most of the metrics show a value of 0.
Any update on this issue?

@zuston
Copy link
Member

zuston commented Jan 14, 2025

Hi,

I am seeing a similar issue where I see most of the metrics show a value of 0. Any update on this issue?

Could you help share some deployment uniffle version and config options? @BaluDontu . From my deployment, the metric system is valid.

@BaluDontu
Copy link

Hi @zuston . I was able to get it resolved. Sorry for the confusion.

I have another question. I am trying to get metric data for these below metrics. I want to understand what is the unit for it?

total_read_data -> Is this in bytes?
total_read_time -> Is this in milliseconds?
total_write_data -> Is this in bytes?
total_write_time -> Is this in milliseconds?
used_buffer_size -> -> Is this in bytes?

Can you point to the documentation about these metrics information?

@zuston
Copy link
Member

zuston commented Jan 16, 2025

total_read_data -> Is this in bytes? yes
total_read_time -> Is this in milliseconds? yes
total_write_data -> Is this in bytes? yes
total_write_time -> Is this in milliseconds? yes
used_buffer_size -> -> Is this in bytes? yes

The unit of the metrics about data size are all bytes. @BaluDontu

@BaluDontu
Copy link

@zuston How about time metrics ?

@zuston
Copy link
Member

zuston commented Jan 16, 2025

@zuston How about time metrics ?

If I'm right, the unit is milliseconds for the time metrics

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants