A gentle introduction to elastic Timelion

Timelion is the latest addition to the Kibana UI with the introduction of Elastic Stack. Timelion is used for analysis and visualizations of time-series data. It provides the ability to combine multiple data sources into a single visualization and gives a range of mathematical calculations that can be used, such as cumulative sum, derivative, moving averages, and so on.

Timelion is present in the left-pane of the Kibana UI between icons of Dashboard and Dev Tools. It has its own language and expressions when using it, which makes it difficult to start with. However, it has a great built-in documentation and tutorial to guide you on how to start using Timelion.

Timelion Expressions

Every Timelion expression starts with a data source function and continues with a chain of functions that are connected with a dot. Over 20 functions are provided, across three groups:
- Data sources: the default is Elasticsearch, and other APIs such as World Bank and Quandl are also available.

For example in the graph below, the default expression .es(*) (similar to .elasticsearch(*)) shows a count of all documents in Elasticsearch. You can specify details of the Elasticsearch index, mappings and metrics here too, as well as filters.
- Data manipulations ranging from simple arithmetic to moving averages, cumulative sums and derivatives.

For example, adding a moving average to the data is as simple as including the function to the end of the expression: .es(*).movingaverage(15)

  • Themes and styles of the visual elements including bar/point/lines, labels, title and legends. For example the graph below monitors memory consumption overtime in a server using data sent by metricbeats:

    .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.pct'), .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.pct').if(gt,0.8,.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.pct'),null).label('warning').color('#FFCC11'), .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.pct').if(gt,0.9,.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.pct'),null).label('severe').color('red')
    

Elastic team improved a little bit timelion documentation by adding some tutorials and examples, but it still honestly somewhat sparse. For details of each function you can refer to the documentation on github. Compared to the rest of the excellent Elastic documentation, this is surprising and hopefully now that Timelion is part of the core product its documentation will be brought up to parity - full explanations of features and functions.

On the positive side, the query builder text box supports auto-complete of functions and their arguments, and the Timelion interface provides online help too. A downside to this minimalist Timelion page is the size of the expression textbox :(

If you are a beginner, to avoid the confusion over typos and errors, try building the expressions step by step and add functions gradually. The blog here and here nicely explains how to gradually create Timelion expressions.