Start a new topic

Monitoring Commander JVM with Prometheus

Monitoring your Commander server is a critical component in ensuring

visibility and insight into the performance and potential stability issues of your environment.


There are many options to gain metrics on Commander. One of the simplest is to monitor the JVM

running Commander using JMX. With a java agent scraping JVM metrics and mBeans you can learn a lot

about what's going on inside your server instance.


In this tutorial we will be using jmx_exporter as our Java scraper agent, and Prometheus as our

time series database to store and view metrics.


Prerequisites:

* A Commander server (any version)

* jmx_exporter (https://github.com/prometheus/jmx_exporter )

* Prometheus (https://prometheus.io/)


Lets install jmx_exporter on your Commander server and configure it as a java agent in tomcat.

Download the jar from jmx_exporter GitHub releases and place it somewhere on your server,

for example in <Commander install>/tomcat/bin. (The jar is also available from the maven.org repository)


To tell the JVM to install the agent you'll need to add a line to the java options.

Open vlmw.exe (found in <Commander install>/tomcat/bin) and go to the Java tab.

Append a line to the "Java Options" field like so:

-javaagent:./jmx_prometheus_javaagent-<version>.jar=9001:config.yaml

(Replacing <version> with the version of jmx_exporter you have)

The configuration file (config.yaml) doesn't have to exist but you can configure advanced options by creating this file.


Hit OK. After this, you'll have to restart the Commander vlm service.

You'll know its working when you can navigate to localhost:9001/metrics in your browser and see a list of metrics 

that the agent has collected. You may have to open your firewall on port 9001 so the Prometheus server can connect to it.


Now you'll want a Prometheus server. There are many ways to deploy Prometheus and the

exact details are outside the scope of this post. For testing purposes though, I recommend 

running a docker container somewhere that can access your Commander server.


docker run -d -p 9090 -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus


Once the container starts up, you should be able to navigate to localhost:9090 and see the Prometheus 

dashboard. But this on its own won't do much. We'll need to configure it.


As part of the scrape configs we'll add this target:

scrape_configs:
- job_name: jmx
  honor_timestamps: true
  scrape_interval: 1m
  scrape_timeout: 30s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - <commander server address>:9001



Make sure that your prometheus instance can access your commander server on port 9001.


If it was successful, you'll see jmx show up in your list of targets (Status->Targets in the menu).

If Prometheus is able to retrieve data from the agent, the State of the target should show "UP".

If it does not, make sure that you can access <commander address>:9001/metrics from a browser or curl, and if that is working,

make sure your scrape configs are correct in prometheus.


Over time, prometheus will collect and store time series metrics from jmx_exporter. You can then use the "Graph" view 

to make some simple queries on this data. 


An example query to view the heap memory usage of Commander would be:

jvm_memory_bytes_used{area="heap"}


Or, for another example, to see how long the JVM is spending in garbage collection:

rate(jvm_gc_collection_seconds_count[5m])


There are lots of available metrics. For the full list see the jmx_exporter documentation.


Collecting data is only part of the story. Next steps would be to set up alerting (such as with alertmanager) 

or visualization (such as with grafana or kibana). Once you have data you can start gaining insight into

your systems bottlenecks and problem areas - giving you the tools you need to diagnose, troubleshoot, and correct

issues.

Login or Signup to post a comment