Import data into InfluxDB

The scenario is:

1. Some process sends metrics to Telegraf 
2. For some reason, Telegraf outputs metrics to a file
3. Import the file from 2) into InfluxDB

 

Telegraf output to a file

Telegraf has a file output plugin, just enable it:

# # Send telegraf metrics to file(s)

 [[outputs.file]]

#   ## Files to write to, "stdout" is a specially handled file.

   files = ["/tmp/metrics.out"]

#

#   ## Data format to output.

   data_format = "influx"

 

Import data into InfluxDB

InfluxDB CLI has a -import argument, the most detailed documentation about how to use it is https://github.com/influxdata/influxdb/tree/master/importer, but it talks about InfluxDB 0.8.9, so that is confusing.  By reading the source code, I figured out the data file has to follow this format (the green lines are my explaination):

# DDL

#if you need to create a database to store your data, put the DDL commands below, e.g.

CREATE DATABASE db0

CREATE DATABASE db1

CREATE RETENTION POLICY rp1 ON db1 DURATION 1h REPLICATION 1

# DML

#specify DB, retention policy for your data below, e.g.

# CONTEXT-DATABASE:db0

# CONTEXT-RETENTION-POLICY:rp1

#and put your data in one-line protacal here, e.g.

mem,host=QAVM107 active=0i,available=15573110784i,available_percent=90.64989902698733 1489567540000000000

I already had the database created, so I just need to insert these lines into the file generated by Telegraf:

# DML

# CONTEXT-DATABASE:test

Now run –import:

influx -import -path=/tmp/metrics.out -precision=s

2017/03/16 15:56:47 error writing batch:  {"error":"unable to parse 

 'mem,host=QAVM107 active=0i,available=15573110784i,available_percent=90.64989902698733 1489567540000000000': 

time outside range -9223372036854775806 - 9223372036854775806"}

This is most strange, to prove that the data point is in the correct format, I inserted it directly:

influx -database telegraf -execute 'insert mem,host=QAVM107 active=0i,available=15573110784i,available_percent=90.64989902698733 1489567540000000000'

Sure this is successful, proving that the data point string is in the correct format and the timestamp is not outside of the time range. 

To find out what is wrong, I enabled all logs in /etc/influxdb/influxdb.conf, and restarted influxdb:

sudo service influxd stop

influxd

Nothing interesting got logged. 

As a last resort, I tried this command without –precision argument and it worked!

influx -import -path=/tmp/metrics.out 

So that is the whole story. Without going into more details in the source code, I can’t tell why –precision argument caused this issue.