Under the Metrics tab, add new metric that will be hidden in the chart and is used for alerting only. Duplicate the query and remove all template variables (i.e. $somevar) from it. Replace the template variable with a hard-coded value you want to create alert for. Hide the metric by clicking on the “eye” icon. … Read more
According to my Grafana dashboard, the following metrics work nicely for alerting for available space, 100 – ((node_filesystem_avail_bytes{mountpoint=”https://stackoverflow.com/”,fstype!=”rootfs”} * 100) / node_filesystem_size_bytes{mountpoint=”https://stackoverflow.com/”,fstype!=”rootfs”}) The formula gives out the percentage of available space on the pointed disk. Make sure you include the mountpoint and fstype within the metrics.
There are a number of options you can add to your ini file to tell Grafana how to build self-referential urls: #################################### Server ############################## [server] # Protocol (http or https) protocol = http # The http port to use http_port = 3000 # The public facing domain name used to access grafana from a browser … Read more
Grafana v5+ provides direct support for representing Prometheus histograms as heatmap. http://docs.grafana.org/features/panels/heatmap/#histograms-and-buckets Heatmaps are preferred over histogram because a histogram does not show you how the trend changes over time. So if you have a time-series histogram, then use the heatmap panel to picture it. To get you started, here is an example (for Prometheus … Read more
There are two ways that I know: You can use the $__interval variable like this: increase(http_requests_total[$__interval]) There is a drawback that the $__interval variable’s value is adjusted by resolution of the graph, but this may also be helpful in some situations. This approach should fit your case better: Go to Dashboard’s Templating settings, create new … Read more
Take a look at Telegraf. It does support tailing logs using input plugins logparser and tail. To export metrics as prometheus endpoint use prometheus_client output plugin. You also may apply on the fly aggregations. I’ve found it simpler to configure for multiple log files than grok_exporter or mtail
Prometheus has an or logical binary operator, but what you’re asking about here is vector selectors. You can use a regex for this {job=~”traefik|cadvisor|prometheus”}, however that you want to do this is a smell.
It uses the query API endpoint rather than the query_range API endpoint on Prometheus, which is more efficient if you only care about the end of your time range and don’t want to pull in data that Grafana is going to throw away again.
New versions got a “Save As…” button in the dashboard menu:
count(count by (a) (hello_info)) First you want an aggregator with a result per value of a, and then you can count them.