How to Install ELK Stack on RHEL 8 / CentOS 8

In this blog post, I’ll show you how to install and configure ELK Stack on RHEL 8 / CentOS 8. “ELK” is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Below is a short description of each tool in the stack.

• Elasticsearch: This is an open source, distributed, RESTful, JSON-based search engine. It is scalable, easy to use, and flexible
• Logstash : This is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a “stash” like Elasticsearch.
• Kibana lets users visualize data with charts and graphs in Elasticsearch.

Our last article covered the installation of Elasticsearch on RHEL 8. For simplicity and convenience, we will cover steps to install all the three ELK stack tools on RHEL 8.

Step 1: Install Java on RHEL 8 / CentOS 8

As Elasticsearch depends on Java, you need it installed on your system prior to installing Elasticsearch 6 or 5 on RHEL 8.

How to Install Java 11 (OpenJDK 11) on RHEL / CentOS 8

Step 2: Add ELK repository to RHEL 8 / CentOS 8

Once you have Java installed, add ELK stack repository to RHEL 8. Please run the commands below as root user:

For Elasticsearch 7.x

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

For Elasticsearch 6.x.

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

For Elasticsearch 5.x.

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

After adding the repo, import GPG key:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Clear and update your YUM package index.

sudo yum clean all
sudo yum makecache

Step 3: Install and Configure Elasticsearch

Elasticsearch repository is ready for use. You can install Elasticsearch using the command below:

sudo yum -y install elasticsearch

Confirm package installation.

$ rpm -qi elasticsearch 
 Name        : elasticsearch
 Epoch       : 0
 Version     : 6.5.4
 Release     : 1
 Architecture: noarch
 Install Date: Wed 02 Jan 2019 02:12:22 PM EAT
 Group       : Application/Internet
 Size        : 167695796
 License     : Elastic License
 Signature   : RSA/SHA512, Tue 18 Dec 2018 01:18:33 AM EAT, Key ID d27d666cd88e42b4
 Source RPM  : elasticsearch-6.5.4-1-src.rpm
 Build Date  : Tue 18 Dec 2018 12:21:58 AM EAT
 Build Host  : packer-virtualbox-iso-1542822913
 Relocations : /usr 
 Packager    : Elasticsearch
 Vendor      : Elasticsearch
 URL         : https://www.elastic.co/
 Summary     : Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
 Description :
 Elasticsearch subproject :distribution:packages

You can set JVM options like memory limits by editing the file: /etc/elasticsearch/jvm.options

Example below sets initial/maximum size of total heap space

-Xms1g
-Xmx1g

If your system has less memory, you can configure it to use small megabytes of ram.

-Xms256m
-Xmx512m

Start and enable elasticsearch service on boot:

$ sudo systemctl enable --now elasticsearch.service 
 Synchronizing state of elasticsearch.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
 Executing: /usr/lib/systemd/systemd-sysv-install enable elasticsearch
 Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /usr/lib/systemd/system/elasticsearch.service.

Test to verify that it is working:

$ curl http://127.0.0.1:9200 
 {
   "name" : "bBzN5Kg",
   "cluster_name" : "elasticsearch",
   "cluster_uuid" : "LKyqXXSvRvCpX9QAwKlP2Q",
   "version" : {
     "number" : "6.5.4",
     "build_flavor" : "default",
     "build_type" : "rpm",
     "build_hash" : "d2ef93d",
     "build_date" : "2018-12-17T21:17:40.758843Z",
     "build_snapshot" : false,
     "lucene_version" : "7.5.0",
     "minimum_wire_compatibility_version" : "5.6.0",
     "minimum_index_compatibility_version" : "5.0.0"
   },
   "tagline" : "You Know, for Search"
 }

Create a test index:

$ curl -X PUT "http://127.0.0.1:9200/mytest_index"
{"acknowledged":true,"shards_acknowledged":true,"index":"mytest_index"}

Step 3: Install and Configure Kibana on RHEL 8 / CentOS 8

Download and install Kibana from added Elasticsearch repository.

sudo yum -y install kibana

After a successful installation, configure Kibana:

$ sudo vim /etc/kibana/kibana.yml
 server.host: "0.0.0.0"
 server.name: "kibana.example.com"
 elasticsearch.url: "http://localhost:9200"

Change other settings as desired then start kibana service:

sudo systemctl enable --now kibana

Access http://ip-address:5601 to open Kibana Dashboard:

If you have an active firewall service, allow TCP port 5601

sudo firewall-cmd --add-port=5601/tcp --permanent
sudo firewall-cmd --reload

Step 4: Install and Configure Logstash on RHEL 8 / CentOS 8

The last installation is for Logstash. It will act as a centralized logs server for your client systems which runs an agent like filebeat.

sudo yum -y install logstash

Logstash custom configurations can be placed under the /etc/logstash/conf.d/ directory. Check Logstash Configuration manual for more details.

Step 5: Install other ELK tools – Bonus

Other ELK tools that can be installed include:

• Filebeat: Lightweight Shipper for Logs. It helps you keep the simple things simple by offering a lightweight way to forward and centralize logs and files
• Metricbeat: Collect metrics from your systems and services. From CPU to memory, Redis to NGINX, and much more, Metricbeat is a lightweight way to send system and service statistics.
• Packetbeat: Lightweight Shipper for Network Data
• Heartbeat: Lightweight Shipper for Uptime Monitoring. It helps you monitor services for their availability with active probing
• Auditbeat: Lightweight shipper that helps you audit the activities of users and processes on your systems

These tools can be installed with yum package manager using their respective names. The example below will install all ELK addon tools.

sudo yum install filebeat auditbeat metricbeat packetbeat heartbeat-elastic

Refer to official ELK stack documentation for each tool configuration. You can also learn online on Resources and Training page by watching videos.