Prometheus is a monitoring system that collects metrics in real time and stores them in a time series database.

Prometheus comes with a console accessible at the 9090 port of the application pod. By default, a template provides an existing service and a route to access it. It can be used to query and view metrics.

monitoring che prometheus console

Prometheus terminology

Prometheus offers:

counter

the simplest numerical type of metric whose value can be only increased. A typical example is counting the amount of HTTP requests that go through the system.

gauge

numerical value that can be increased or decreased. Best suited for representing values of objects.

histogram

a more complex metric that is suited for performing observations. Metrics are collected and grouped in configurable buckets, which allwos to present the results, for instance, in a form of a heatmap.

Configuring Prometheus

Prometheus configuration
- apiVersion: v1
  data:
    prometheus.yml: |-
      global:
        scrape_interval:     5s           (1)
        evaluation_interval: 5s           (2)
      scrape_configs:
        - job_name: 'che'
          static_configs:
            - targets: ['che-host:8087']
  kind: ConfigMap
  metadata:
    name: prometheus-config
1 rate, at which a target is scraped
2 rate, at which recording and alerting rules are re-checked (not used in our system at the moment)