Graylog is an opensource log aggregation and management tool which can be used to store, analyse and send alerts from the logs collected. Graylog can be used to analyse both structured and unstructured logs using ElasticSearch and MongoDB. This includes a variety of systems including Windows systems, Linux systems, different applications and micro-services etc.
Graylog makes it easier to easily analyse, and monitor these systems and applications from a single host.
Graylog has the following components:
- Graylog server
- MongoDB
- ElasticSearch
Let us quickly jump into the installation of Graylog server on an Ubuntu 20.04 host. We shall then configure SSL using Let’sEncrypt.
To achieve this, we will need to install Nginx to serve as a reverse-proxy on our system.
Prerequisites
Before we can install on your box, make sure your host meets the below minimal requirements:
- 4 CPU Cores
- 8 GB RAM
- SSD Hard Disk Space with High IOPS for Elasticsearch Log Storage
- Ubuntu 20.04 LTS installed and updated.
- All packages upgraded
With the above conditions met, let us begin the installation process.
Step 1 – Install Java on Ubuntu 20.04
Java version 8 and above is required for Graylog installation.
In this post, we shall use open JDK 11
sudo apt update
sudo apt install -y apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen curl dirmngrYou can verify the java version you just installed using the java -version command:
$ java -version
openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment (build 11.0.9+11-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.9+11-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)Step 2 – Install Elasticsearch on Ubuntu 20.04
Elastic search is the tool that is used to store and analyse incoming logs from external sources. It uses the web-based RESTful API.
Download and install Elasticsearch GPG signing key.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -Add Elasticsearch repository to your sources list:
echo "deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-6.x.listInstall Elasticsearch:
sudo apt update
sudo apt install -y elasticsearch-ossConfigure cluster name for Graylog.
sudo vim /etc/elasticsearch/elasticsearch.ymlEdit the cluster name to graylog
cluster.name: graylogAdd the following information in the same file
action.auto_create_index: falseReload daemon the start Elasticsearch service.
sudo systemctl daemon-reload
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearchYou can check for the status of the service by :
$ systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-11-08 12:36:10 UTC; 14s ago
Docs: http://www.elastic.co
Main PID: 1352139 (java)
Tasks: 15 (limit: 4582)
Memory: 1.1G
CGroup: /system.slice/elasticsearch.service
└─1352139 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.>
Nov 08 12:36:10 graylog.computingforgeeks.com systemd[1]: Started Elasticsearch.Elasticsearch runs on port 9200 and this can be virified by curl command:
curl -X GET http://localhost:9200You should see your cluster name in the output.
$ curl -X GET http://localhost:9200
{
"name" : "RQ7y68X",
"cluster_name" : "graylog",
"cluster_uuid" : "Dp948OGjT-uFz_LuOiW6ng",
"version" : {
"number" : "6.8.13",
"build_flavor" : "oss",
"build_type" : "deb",
"build_hash" : "be13c69",
"build_date" : "2020-10-16T09:09:46.555371Z",
"build_snapshot" : false,
"lucene_version" : "7.7.3",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}Step 3 – Install MongoDB on Ubuntu 20.04
Download and install mongoDB from Ubuntu’s base repository.
sudo apt update
sudo apt install -y mongodb-serverStart MongoDB
sudo systemctl start mongodb
sudo systemctl enable mongodb$ systemctl status mongodb
● mongodb.service - An object/document-oriented database
Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-11-08 12:45:21 UTC; 1s ago
Docs: man:mongod(1)
Main PID: 1352931 (mongod)
Tasks: 3 (limit: 4582)
Memory: 27.9M
CGroup: /system.slice/mongodb.service
└─1352931 /usr/bin/mongod --unixSocketPrefix=/run/mongodb --config /etc/mongodb.conf
Nov 08 12:45:21 graylog.computingforgeeks.com systemd[1]: Started An object/document-oriented database.
Step 4 – Install Graylog Server on Ubuntu 20.04
Download and configure Graylog repository.
wget https://packages.graylog2.org/repo/packages/graylog-3.3-repository_latest.deb
sudo apt install ./graylog-3.3-repository_latest.debInstall Graylog server:
sudo apt update
sudo apt install -y graylog-serverGenerate a secret to secure user passwords using pwgen command
pwgen -N 1 -s 96The output should look like:
FFP3LhcsuSTMgfRvOx0JPcpDomJtrxovlSrbfMBG19owc13T8PZbYnH0nxyIfrTb0ANwCfH98uC8LPKFb6ZEAi55CvuZ2AumEdit the graylog config file to add the secret we just created:
sudo vim /etc/graylog/server/server.confLocate the password_secret = line and add the above created secret after it.
password_secret= FFP3LhcsuSTMgfRvOx0JPcpDomJtrxovlSrbfMBG19owc13T8PZbYnH0nxyIfrTb0ANwCfH98uC8LPKFb6ZEAi55CvuZ2AumAlso add the following lines to the /etc/graylog/server/server.conf file
rest_listen_uri = http://127.0.0.1:9000/api/
web_listen_uri = http://127.0.0.1:9000/The next step is to create a hash sha256 pasword for the administrator. This is the password you will need to login to the web interface.
echo -n Str0ngPassw0rd | sha256sumReplace ‘Str0ngPassw0rd’ with a password of your choice.
You will get an output of this kind:
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951Edit the /etc/graylog/server/server.conf file then place the hash password at root_password_sha2 =
sudo vi /etc/graylog/server/server.confroot_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951Graylog is now configured and ready for use.
Start Graylog service:
sudo systemctl daemon-reload
sudo systemctl start graylog-server
sudo systemctl enable graylog-serverYou can check if the service has started successfully from the logs:
sudo tail -f /var/log/graylog-server/server.logOutput:
2020-11-08T13:37:55.067Z INFO [ServerBootstrap] Graylog server up and running.Step 5 – Setup SSL using Let’s Encrypt
The next step is to configure SSL so that we can access Graylog web interface via HTTPS.
To achieve this, we will need the following:
- Fully qualified domain name(FQDN)
- Nginx
- Let’sEncrypt certificate
Use the steps below to install and configure ngiinx.
- update system and install nginx
sudo apt-get update
sudo apt install nginx2. Configure firewall
sudo ufw allow 'Nginx Full'3. Create virtualhost with your domain name
Create a file in /etc/nginx/sites-available/ e.g
sudo vim /etc/nginx/sites-available/graylog.yourdomain.com.confAdd the following in the file:
server {
listen 80;
server_name graylog.yourdomain.com;
return 301 https://$host$request_uri;
access_log /var/log/nginx/graylog.yourdomain.com.access.log combined;
error_log /var/log/nginx/graylog.yourdomain.com.error.log;
}
Remember to substitute graylog.yourdomain.com with your FQDN.
4. Create a symlink of the file we just created in /etc/nginx/sites-available to /etc/nginx/sites-enabled
sudo ln -s /etc/nginx/sites-available/graylog.yourdomain.com.conf /etc/nginx/sites-enabled/5. Check if nginx config is okay by running nginx -t command.
$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful6. Install Let’sEncrypt with certbot.
sudo apt install certbot python3-certbot-nginx7. Run certbot for nginx
$ sudo certbot --nginx
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel): [email protected]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: A
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: N
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: graylog.computingforgeeks.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for graylog.computingforgeeks.com
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/graylog.computingforgeeks.com.conf
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/graylog.computingforgeeks.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled
https://graylog.computingforgeeks.com
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=graylog.computingforgeeks.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/graylog.computingforgeeks.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/graylog.computingforgeeks.com/privkey.pem
Your cert will expire on 2021-02-06. To obtain a new or tweaked
version of this certificate in the future, simply run certbot again
with the "certonly" option. To non-interactively renew *all* of
your certificates, run "certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
You have successfully obtained SSL for our domain.
The next step is to configure reverse proxy on Nginx that will be used to serve Graylog which is running on the same host on port 9000.
Edit the /etc/nginx/sites-available/graylog.yourdomain.com.conf file and add the following configuration under the Location section.
location /
{
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL https://$server_name/;
proxy_pass http://127.0.0.1:9000;
}
The final config file should look like:
server {
listen 80;
server_name graylog.computingforgeeks.com;
return 301 https://$host$request_uri;
access_log /var/log/nginx/graylog.computingforgeeks.com.access.log combined;
error_log /var/log/nginx/graylog.computingforgeeks.com.error.log;
}
server {
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name graylog.computingforgeeks.com;
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/graylog.computingforgeeks.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/graylog.computingforgeeks.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
location /
{
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL https://$server_name/;
proxy_pass http://127.0.0.1:9000;
}
}
Verify your nginx configuration using the nginx -t just to make sure your nginx configuration is okay.
Now restart nginx service.
sudo systemctl restart nginxWith the above up and running, you should be able to access your graylog dashboard by entering https://graylog.yourdomain.com.
Remember to replace graylog.yourdomain.com with your FQDN.
The default username for Graylog is admin and the password we configured in step 4 (Install Graylog server) above. For my case, this would be “Str0ngPassw0rd”
You can now start using your Graylog web dashboard configured with SSL.
Conclusion
We have successfully installed graylog server, configured SSL through Nginx as a reverse proxy and managed to login to the web interface.
Configuring SSL on Graylog server is important for securing your system.
Should you face any challenge during the setup process feel free to comment or ask any question on the comments section.
Configure Graylog Nginx reverse proxy with Letsencrypt SSL
