(Last Updated On: May 2, 2018)

This will be our first series on how to do smart monitoring and visualization of your entire Infrastructure. The entire series will concentrate on tools like Grafana, Prometheus, InfluxDB, Telegraf and maybe others to come.

Since this is the first article in the series, I’ll make an introductory definition of tools that we’ll be working with:

InfluxDB is an open-source time series database developed written in Go by InfluxData. InfluxDB is optimized for fast, high-availability storage and retrieval of time series data for metrics analysis. This can be installed on a single server or a clustered.

Telegraf is an agent written in Go for collecting, processing, aggregating, and writing metrics. Design goals are to have a minimal memory footprint with a plugin system so that developers in the community can easily add support for collecting metrics from local or remote services. It is installed on all devices that need to be monitored, and all metrics collected by Telegraf are pushed stored on InfluxDB.

Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus, and InfluxDB. Data stored on InfluxDB will be visualized using Grafana.

Prometheus is a tool used for systems and service monitoring. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true. Metrics stored on Prometheus can be visualized using Grafana.

Installing InfluxDB on CentOS 7

Influxdata provides the repository for installing InfluxDB and telegraf on CentOS 7. To add the repository to your system, use the commands:

vim  /etc/yum.repos.d/influxdb.repo

Add the following content:

[influxdb]
name = InfluxDB Repository - RHEL \$releasever
baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key

Update cache to confirm that the repository is working fine:

$ sudo yum makecache fast

Then install influxDB:

# yum -y install influxdb

Start and enable the service:

# systemctl start influxdb && systemctl enable influxdb

Configure InfluxDB firewall

By default, InfluxDB uses the following network ports:

TCP port 8086 is used for client-server communication over InfluxDB’s HTTP API
TCP port 8088 is used for the RPC service for backup and restore.

To open it on the firewall, use the command:

# firewall-cmd --add-port=8086/tcp --permanent
# firewall-cmd --reload

Port mappings can be modified by changing the file /etc/influxdb/influxdb.conf. Since all is configured now, we can start the service now.

# systemctl start influxdb && systemctl enable influxdb

InfluxDB http Authentication

If you need http authentication, modify influxdb http section to contain the following.

[http]
auth-enabled = true

Then create a user with authentication password:

# curl -XPOST "http://localhost:8086/query" --data-urlencode "q=CREATE USER \
username WITH PASSWORD 'strongpassword' WITH ALL PRIVILEGES"

Replace:

username with your own username
strongpassword with your own password (note that the password requires single quotes)

Now whenever you need to run any influxdb commands on the terminal, you need to specify username using -username and password using -password options.

# influx -username 'username' -password 'password'

For curl, use -u to specify username and password separated by a colon.

# curl -G http://localhost:8086/query -u username:password --data-urlencode "q=SHOW DATABASES"

By default, influxdb service is listening on all interfaces on port 8086.

Installing Grafana on CentOS 7

There are two ways to install Grafana on CentOS 7, one is using official Grafana yum repository, and the other method involves manually downloading rpm package and installing it locally on the server.

The preferred method is using repo since it is easy to update to latest release. So add the following to a new file at /etc/yum.repos.d/grafana.repo

[grafana]
name=grafana
baseurl=https://packagecloud.io/grafana/stable/el/7/$basearch
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

You’ll be prompted to accept gpg key, press Yes to continue. To start grafana service and enable it to start on boot, run:

# systemctl start grafana-server
# systemctl enable grafana-server

This will start the grafana-server process as the grafana user, which is created during package installation. The default HTTP port is 3000, and default user and group is admin.  By default Grafana will log to /var/log/grafana.

The default configuration  file is /etc/grafana/grafana.in with sqlite3 database store  located at /var/lib/grafana/grafana.db

Open firewall port for Grafana

If you have a running firewalld service, consider opening port 3000 of you’re going to access the dashboard over the network.

# firewall-cmd --add-port=3000/tcp --permanent
# firewall-cmd --reload

You can the access the dashboard on the web using port 3000 and IP address or hostname and start creating Dashboards.

On our next article, we’ll look at how to install telegraf client agent for InfluxDB and collect server metrics for visualization on Grafana.