Both Zabbix and Prometheus may be used in various monitoring scenarios, and there isn’t any particular specialization in either of these. Zabbix is older than Prometheus and probably more stable, with more ready-to-use solutions. Zabbix has a core written in C and a web UI based on PHP. Also it uses “agents” (client-side programs) written … Read more
The endpoint for that is http://localhost:9090/api/v1/label/__name__/values API Reference
You currently can’t configure the metrics_path per target within a job but you can create separate jobs for each of your targets so you can define metrics_path per target. Your config file would look something like this: scrape_configs: – job_name: ‘example-target-1’ scrape_interval: 5s metrics_path: /target-1-path-to-metrics static_configs: – targets: [‘localhost:8090’] labels: group: ‘dummy’ – job_name: ‘example-target-2’ … Read more
Prometheus has an or logical binary operator, but what you’re asking about here is vector selectors. You can use a regex for this {job=~”traefik|cadvisor|prometheus”}, however that you want to do this is a smell.
It uses the query API endpoint rather than the query_range API endpoint on Prometheus, which is more efficient if you only care about the end of your time range and don’t want to pull in data that Grafana is going to throw away again.
Rate returns a per second value, so multiplying by 100 will give a percentage: rate(container_cpu_user_seconds_total[30s]) * 100
Probably the default scrape_timeout value is too short for you [ scrape_timeout: <duration> | default = 10s ] Set a bigger value for scrape_timeout. scrape_configs: – job_name: ‘prometheus’ scrape_interval: 5m scrape_timeout: 1m Take a look here https://github.com/prometheus/prometheus/issues/1438
Edit the prometheus.service file vi /etc/systemd/system/prometheus.service add “–storage.tsdb.retention.time=1y” below to “ExecStart=/usr/local/bin/prometheus \” line. So the config will look like bellow for 1 year of data retention. [Unit] Description=Prometheus Wants=network-online.target After=network-online.target [Service] User=prometheus Group=prometheus Type=simple ExecStart=/usr/local/bin/prometheus \ –config.file /etc/prometheus/prometheus.yml \ –storage.tsdb.path /var/lib/prometheus/ \ –web.console.templates=/etc/prometheus/consoles \ –web.console.libraries=/etc/prometheus/console_libraries \ –web.external-url=http://34.89.26.156:9090 \ –storage.tsdb.retention.time=1y [Install] WantedBy=multi-user.target
You can use the argument list of group_left to include extra labels from the right operand (parentheses and indents for clarity): ( max(consul_health_service_status{status=”critical”}) by (service_name,status,node) == 1 ) + on(service_name,node) group_left(env) ( 0 * consul_service_tags ) The important part here is the operation + on(service_name,node) group_left(env): the + is “abused” as a join operator (fine … Read more
This I’m using to get CPU usage at cluster level: sum (rate (container_cpu_usage_seconds_total{id=”https://stackoverflow.com/”}[1m])) / sum (machine_cpu_cores) * 100 I also track the CPU usage for each pod. sum (rate (container_cpu_usage_seconds_total{image!=””}[1m])) by (pod_name) I have a complete kubernetes-prometheus solution on GitHub, maybe can help you with more metrics: https://github.com/camilb/prometheus-kubernetes