WSO2EI JVM Monitoring with ELK

Here is to visualize JVM statistics using Kibana dashboard. WSO2EI is running on JVM and ELK setup should be already implemented.

Below statistic will be available in Kibana after this integration

1. Thread count
2. Class count
3. Heap memory
4. Non-heap memory

Tested Environment:

wso2ei-6.1.1

jdk1.8.0_172-amd64

nrpe  3.2.1

filebeat 6.8.0

logstash 6.8.0

ElasticSearch 6.8.0

Kibana 6.8.0

Step 1:

Install check_jvm plugin in the VM as mentioned in below article,

nrpe plugin for JVM

Step 2:

Make sure that you have installed NRPE plugin successfully. Execute below commands and check output

Command

       
/usr/local/nagios/libexec/check_jvm -u igwstuser -n org.wso2.carbon.bootstrap.Bootstrap -p heap -w 2008096384 -c 3000218931
       

Output

       
OK - 826M |max=9544663040;commited=4462215168;used=865669640;
       

Command

       
/usr/local/nagios/libexec/check_jvm -u igwstuser -n org.wso2.carbon.bootstrap.Bootstrap -p non-heap -w 268435456 -c 300870912
       

Output

       
WARNING 267M |max=-1;;; commited=352649216;;;used=279245856;;;
       

Command

       
/usr/local/nagios/libexec/check_jvm -u igwstuser -n org.wso2.carbon.bootstrap.Bootstrap -p classes -w 25000 -c 30000
       

Output

       
OK - 22816 |classes=22816;;;
       

Command

       
/usr/local/nagios/libexec/check_jvm -u igwstuser -n org.wso2.carbon.bootstrap.Bootstrap -p threads -w 700 -c 800
       

Output

       
OK - 655 |threads=655;;;
       

Don’t worry about the warning and critical threshold values, here we are considering values only.

Important: You should execute the above commands from the user who is running WSO2EI.

Step 3:

Generate shell script to execute the above commands and write output to a log file

check_jvm.sh

Step 4:

Add cronjob to execute the above shell script every 2 minutes.

       
*/2 * * * * cd /log/jvm-scripts; sh check_jvm.sh
       

Now data will be collected on the given location.

Step 5:

Here I’m assuming that ELK stack is already installed and configured. I will mention the configurations which are related to this topic only. You can refer below for filebeat and logstash configurations. make sure to restart filebeat and logstash service after adding new configurations.

Filebeat : read log file and push to logstash, please refer filebeat.yml in the below link.

Logstash : read the logs sent by filebeat, match to a correct index after applying to grok patterns  and then puch to elasticsearch, please refer logstash.conf in below link

jvm-monitor-elk

Step 6:

Go to Kibana portal à Management à Elasticsearch à Index Management

Search for jvm and then you should be able to see relevant indexes were created in Elasticsearch  

Step 7:

Go to Kibana à Index patterns à Create index pattern à type “jvm*” and create index pattern.

Step 8:

Now you can create your own visualizations and dashboard to monitor JVM statistics.

You can download the Kibana visualizations here.

Feel free to comment here if anything.