Get to know Kibana - Again

When Kibana was in 3, I played with it quite extensively – you can see the story of my customization in my blog. Since then, the relentless step of software technology has evolved Kibana to 4, and then to 5. Kibana 4 was a complete rewrite of 3. 

I’ve putting off working on customizing Kibana for a long time because my AngularJs skills are very poor, but these days, I finally plucked up courage to do so, which is a good timing, because Kibana 5 has a plugin structure that supports customizations, there is little document about how to develop plugins though. 

First thing is first, Kibana has changed, so let us learn how to use it – once again. 

To use Kibana, follow these steps:

• Configure Elasticsearch url

•  Configure the Index Pattern

• Configure Discover

• Configure Visualize

• Configure Dashboard

Configure Elasticsearch url

This is easy, just open config/kibana.yml and change elasticsearch.url.

Configure the Index Pattern

Clicking on the Settings tab (on the left), and select a time-stamp field:

  

Configure Discover

“Discover” might seems confusing, but what it really does is just narrow down indices with criteria.
 

Note, you should use capitalized “AND” “OR” to define criteria. All fields will be returned, you can decide how to order them in the page by selecting from “Available Fields” (on the left). 

And also note, if you do not see any data, check the time setup in the right upper corner, Kibana will search indices within the time period.  

Once you are done, clicking on Save to save the discovery, you can use the saved discovery in Visualize.

Configure Visualize (date histogram)

For my purpose, I want to create a date histogram, so click Vertical Bar Chart:

 

Select the saved discovery from the last step:

Leave the Y-Axis as it is, and click on X-Axis:

Choose “Date Histogram” and a field that is of “date” type and then click “Add sub-buckets”: 

 

Click on “split bars”:

 

Select “Terms” as the sub aggregation, and a field to aggregate, and click on the little right-facing triangle button, you should be able to see a chart. In my case, I can see how events are aggregated over time. (event is a field my logstash program created, which is a categorization of important logs, for example, an “OutOfMemoryError” with a long stacktrace will be categorized as event OOM. Logs are typically very long and verbose, this categorization serves to distill data and make it easy to see important logs):

 

Now save the Visualize, which will be used in Dashboard.

Configure Dashboard

Click add, and select saved visualizes, and you are done.

 

Sounds complex? Yes, that is why I liked Kibana3 better. But that is how Kibana4 does, we will have to cope with it. My solution is to write a plugin to automatically create Discovery, Visualize and Dashboard. My requirement is to analyze logs from different customers, each customer has multiple nodes, every customer should have a separate dashboard, which shows all nodes’ important logs.

This is the plugin:

 

What it does is: users enters a customer name (“aaa” in the screenshot), clicks on “AutoSetup”, and it will automatically create Discover, Visualization and Dashboard.

I will share how to develop Kibana plugins in my next blog.