Grafana is used for informative representation of Prometheus metrics. Providing visibility for OpenShift, Grafana’s deployment configuration and ConfigMaps are located in the che-monitoring.yaml configuration file.

Configuring and deploying Grafana

Grafana is run on port 3000 with a corresponding service and route.

Three ConfigMaps are used to configure Grafana:

  • grafana-datasources — configuration for Grafana datasource, a Prometheus endpoint

  • grafana-dashboards — configuration of Grafana dashboards and panels

  • grafana-dashboard-provider  — configuration of the Grafana dashboard provider API object, which tells Grafana where to look in the file system for pre-provisioned dashboards

Grafana dashboards overview

Che provides several types of dashboards.

Che server dashboard

Use case: Che server-specific metrics related to Che components, such as workspaces or users.

monitoring che che server dashboard general panel
Figure 1. The General panel

The General panel contains basic information, such as the total number of users and workspaces in the Che database.

monitoring che che server dashboard workspace panel
Figure 2. The Workspaces panel
  • Workspace start rate — the ratio between successful and failed started workspaces

  • Workspace stop rate — the ratio between successful and failed stopped workspaces

  • Workspace Failures — the number of workspace failures shown on the graph

  • Starting Workspaces — the gauge that shows the number of currently starting workspaces

  • Average Workspace Start Time — 1-hour average of workspace starts or fails

  • Average Workspace Stop Time — 1-hour average of workspace stops

  • Running Workspaces — the gauge that shows the number of currently running workspaces

  • Stopping Workspaces — the gauge that shows the number of currently stopping workspaces

  • Workspaces started under 60 seconds — the percentage of workspaces started under 60 seconds

  • Number of Workspaces — the number of workspaces created over time

monitoring che che server dashboard users panel
Figure 3. The Users panel
  • Number of Users — the number of users known to Che over time

monitoring che che server dashboard tomcat panel
Figure 4. The Tomcat panel
  • Max number of active sessions — the max number of active sessions that have been active at the same time

  • Number of current active sessions — the number of currently active sessions

  • Total sessions — the total number of sessions

  • Expired sessions — the number of sessions that have expired

  • Rejected sessions — the number of sessions that were not created because the maximum number of active sessions was reached

  • Longest time of an expired session — the longest time (in seconds) that an expired session had been alive

monitoring che che server dashboard requests panel
Figure 5. The Request panel

The Requests panel displays HTTP requests in a graph that shows the average number of requests per minute.

monitoring che che server dashboard executors panel 1
Figure 6. The Executors panel, part 1
  • Threads running - the number of threads that are not terminated aka alive. May include threads that are in a waiting or blocked state.

  • Threads terminated - the number of threads that was finished its execution.

  • Threads created - number of threads created by thread factory for given executor service.

  • Created thread/minute - Speed of thread creating for the given executor service.

monitoring che che server dashboard executors panel 2
Figure 7. The Executors panel, part 2
  • Executor threads active - number of threads that actively execute tasks.

  • Executor pool size - number of threads that actively execute tasks.

  • Queued task - the approximate number of tasks that are queued for execution

  • Queued occupancy - the percent of the queue used by the tasks that is waining for execution.

monitoring che che server dashboard executors panel 3
Figure 8. The Executors panel, part 3
  • Rejected task - the number of tasks that were rejected from execution.

  • Rejected task/minute - the speed of task rejections

  • Completed tasks - the number of completed tasks

  • Completed tasks/minute - the speed of task execution

monitoring che che server dashboard executors panel 4
Figure 9. The Executors panel, part 4
  • Task execution seconds max - 5min moving maximum of task execution

  • Tasks execution seconds avg - 1h moving average of task execution

  • Executor idle seconds max - 5min moving maximum of executor idle state.

  • Executor idle seconds avg - 1h moving average of executor idle state.

monitoring che che server dashboard trace panel 1
Figure 10. The Traces panel, part 1
  • Workspace start Max - maximum workspace start time

  • Workspace start Avg - 1h moving average of the workspace start time components

  • Workspace stop Max - maximum of workspace stop time

  • Workspace stop Avg - 1h moving average of the workspace stop time components

monitoring che che server dashboard trace panel 2
Figure 11. The Traces panel, part 2
  • OpenShiftInternalRuntime#start Max - maximum time of OpenShiftInternalRuntime#start operation

  • OpenShiftInternalRuntime#start Avg - 1h moving average time of OpenShiftInternalRuntime#start operation

  • Plugin Brokering Execution Max - maximum time of PluginBrokerManager#getTooling operation

  • Plugin Brokering Execution Avg - 1h moving average of PluginBrokerManager#getTooling operation

monitoring che che server dashboard trace panel 3
Figure 12. The Traces panel, part 3
  • OpenShiftEnvironmentProvisioner#provision Max - maximum time of OpenShiftEnvironmentProvisioner#provision operation

  • OpenShiftEnvironmentProvisioner#provision Avg -1h moving average of OpenShiftEnvironmentProvisioner#provision operation

  • Plugin Brokering Execution Max - maximum time of PluginBrokerManager#getTooling components execution time

  • Plugin Brokering Execution Avg - 1h moving average of time of PluginBrokerManager#getTooling components execution time

monitoring che che server dashboard trace panel 4
Figure 13. The Traces panel, part 4
  • WaitMachinesStart Max - maximim time of WaitMachinesStart operations

  • WaitMachinesStart Avg - 1h moving average time of WaitMachinesStart operations

  • OpenShiftInternalRuntime#startMachines Max - maximim time of OpenShiftInternalRuntime#startMachines operations

  • OpenShiftInternalRuntime#startMachines Avg - 1h moving average of the time of OpenShiftInternalRuntime#startMachines operations

monitoring che che server dashboard workspace detailed panel
Figure 14. The Workspace detailed panel

The Workspace Detailed panel contains heat maps, which illustrate the average time of workspace starts or fails. The row shows some period of time.

Che server JVM dashboard

Use case: JVM metrics of the Che server, such as JVM memory or classloading.

monitoring che che server jvm dashboard
Figure 15. Che server JVM dashboard
monitoring che che server jvm dashboard quick facts
Figure 16. Quick Facts
monitoring che che server jvm dashboard jvm memory
Figure 17. JVM Memory
monitoring che che server jvm dashboard jvm misc
Figure 18. JVM Misc
monitoring che che server jvm dashboard jvm memory pools heap
Figure 19. JVM Memory Pools (heap)
monitoring che che server jvm dashboard jvm memory pools non heap
Figure 20. JVM Memory Pools (Non-Heap)
monitoring che che server jvm dashboard garbage collection
Figure 21. Garbage Collection
monitoring che che server jvm dashboard classloading
Figure 22. Classloading
monitoring che che server jvm dashboard buffer pools
Figure 23. Buffer Pools