In this guide, I’m going to show you how to configure MongoDB Replica Set on Ubuntu 18.04 server. MongoDB is an enterprise-class NoSQL database system with auto-scaling, high availability and high performance. In a NoSQL database, data is stored in a document structure using MongoDB BSON format. SQL statements cannot be used in MongoDB to insert or retrieve data.
What is MongoDB Replication / Replica set?
In MongoDB, a replica set is a group of mongod processes that maintain the same data set. Replica sets are the basis for all production deployments as they provide data redundancy and high availability.
This tutorial will describe the steps required to create a working three nodes replica set without configured access control to restrict access. This guide is based on below system IP Addresses:
Node1: 10.10.5.2 Node2: 10.10.5.3 Node3: 10.10.5.4
Step 1: Setup Requirements
Before you can configure a replica set, you must install MongoDB on each system that will be part of the replica set. Set correct hostnames for all the servers. If you have an active DNS server, add A records for all servers, or modify /etc/hosts file. Add these on all nodes.
$ sudo vim /etc/hosts 10.10.5.2 mongodb-01 10.10.5.3 mongodb-02 10.10.5.4 mongodb-03
FYI: My Lab Setup is based on below Vagrantfile running on KVM.
[pastacode lang=”bash” manual=”%23%20-*-%20mode%3A%20ruby%20-*-%0A%23%20vim%3A%20set%20ft%3Druby%20%3A%0A%0AENV%5B’VAGRANT_DEFAULT_PROVIDER’%5D%20%3D%20’libvirt’%0A%0A%23%20Check%20required%20plugins%0AREQUIRED_PLUGINS_LIBVIRT%20%3D%20%25w(vagrant-libvirt)%0Aexit%20unless%20REQUIRED_PLUGINS_LIBVIRT.all%3F%20do%20%7Cplugin%7C%0A%20%20Vagrant.has_plugin%3F(plugin)%20%7C%7C%20(%0A%20%20%20%20puts%20%22The%20%23%7Bplugin%7D%20plugin%20is%20required.%20Please%20install%20it%20with%3A%22%0A%20%20%20%20puts%20%22%24%20vagrant%20plugin%20install%20%23%7Bplugin%7D%22%0A%20%20%20%20false%0A%20%20)%0Aend%0A%0AVagrant.configure(%222%22)%20do%20%7Cconfig%7C%0A%20%20config.vm.define%20%22mongodb-01%22%20do%20%7Cnode%7C%0A%20%20%20%20node.vm.hostname%20%3D%20%22mongodb-01%22%0A%20%20%20%20node.vm.box%20%3D%20%22generic%2Fubuntu1804%22%0A%20%20%20%20node.vm.box_check_update%20%3D%20false%0A%20%20%20%20%23node.vm.synced_folder%20′.’%2C%20’%2Fvagrant’%2C%20%3Adisabled%20%3D%3E%20true%0A%20%20%20%20node.vm.network%20%22private_network%22%2C%20ip%3A%20%2210.10.5.2%22%0A%20%20%20%20node.vm.provider%20%3Alibvirt%20do%20%7Cdomain%7C%0A%20%20%20%20%20%20domain.memory%20%3D%20512%0A%20%20%20%20%20%20domain.storage%20%3Afile%2C%20%3Asize%20%3D%3E%20’10G’%0A%20%20%20%20end%0A%20%20end%0A%20%20config.vm.define%20%22mongodb-02%22%20do%20%7Cnode%7C%0A%20%20%20%20node.vm.hostname%20%3D%20%22mongodb-02%22%0A%20%20%20%20node.vm.box%20%3D%20%22generic%2Fubuntu1804%22%0A%20%20%20%20node.vm.box_check_update%20%3D%20false%0A%20%20%20%20%23node.vm.synced_folder%20′.’%2C%20’%2Fvagrant’%2C%20%3Adisabled%20%3D%3E%20true%0A%20%20%20%20node.vm.network%20%22private_network%22%2C%20ip%3A%20%2210.10.5.3%22%0A%20%20%20%20node.vm.provider%20%3Alibvirt%20do%20%7Cdomain%7C%0A%20%20%20%20%20%20domain.memory%20%3D%20512%0A%20%20%20%20%20%20domain.storage%20%3Afile%2C%20%3Asize%20%3D%3E%20’10G’%0A%20%20%20%20end%0A%20%20end%0A%20%20config.vm.define%20%22mongodb-03%22%20do%20%7Cnode%7C%0A%20%20%20%20node.vm.hostname%20%3D%20%22mongodb-03%22%0A%20%20%20%20node.vm.box%20%3D%20%22generic%2Fubuntu1804%22%0A%20%20%20%20node.vm.box_check_update%20%3D%20false%0A%20%20%20%20%23node.vm.synced_folder%20′.’%2C%20’%2Fvagrant’%2C%20%3Adisabled%20%3D%3E%20true%0A%20%20%20%20node.vm.network%20%22private_network%22%2C%20ip%3A%20%2210.10.5.4%22%0A%20%20%20%20node.vm.provider%20%3Alibvirt%20do%20%7Cdomain%7C%0A%20%20%20%20%20%20domain.memory%20%3D%20512%0A%20%20%20%20%20%20domain.storage%20%3Afile%2C%20%3Asize%20%3D%3E%20’10G’%0A%20%20%20%20end%0A%20%20end%0Aend%0A” message=”” highlight=”” provider=”manual”/]
I then ran vagrant up to start all VMs
Step 2: Install MongoDB on all Ubuntu 18.04 Nodes
If you have not already installed MongoDB, see our installation guide How to install Latest MongoDB on Ubuntu 18.04 / Ubuntu 16.04. I will include installation guide here for convenience.
Update system:
sudo apt-get update sudo apt-get upgrade
Import MongoDB public GPG Key:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
Once the key has been imported, add the repository:
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list
For my setup, I have 10GB secondary storage dedicated for MongoDB data. I’ll partition it and mount to /data/mongodb.
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 32G 0 disk |-sda1 8:1 0 487M 0 part /boot |-sda2 8:2 0 1.9G 0 part [SWAP] `-sda3 8:3 0 29.6G 0 part / vda 252:0 0 10G 0 disk
Create a GPT partition table for the secondary disk, it can be more than one disk
parted -s -a optimal -- /dev/vda mklabel gpt parted -s -a optimal -- /dev/vda mkpart primary 0% 100% parted -s -- /dev/vda align-check optimal 1
Then create LVM volume, this will make it easy to extend the partition
# pvcreate /dev/vda1 Physical volume "/dev/vda1" successfully created. # vgcreate vg11 /dev/vda1 Volume group "vg11" successfully created # lvcreate -n data -l 100%FREE vg11 Logical volume "data" created
Create a ext4filesystem on the Logical Volume created.
# mkfs.ext4 /dev/mapper/vg11-data
mke2fs 1.44.1 (24-Mar-2018)
Creating filesystem with 2620416 4k blocks and 655360 inodes
Filesystem UUID: b98e07e5-1b04-4282-a9db-fa5b73c74d2f
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
Create a mount point and mount the partition
echo "/dev/mapper/vg11-data /data ext4 defaults 0 0" >> /etc/fstab mkdir /data mount -a
Confirm that the partition mount was successful:
# df -hT | grep /data /dev/mapper/vg11-data ext4 9.8G 37M 9.3G 1% /data
Install MongoDB packages
sudo apt-get update sudo apt-get install -y openssl libcurl3 mongodb-org
Create a folder for MongoDB data
mkdir /data/mongodb chown -R mongodb:mongodb /data/mongodb chmod -R 775 /data/mongodb
Step 3: Configure MongoDB Replica set
Now that we have everything needed ready, let’s proceed to configure MongoDB replica set.
Change MongoDB Listen Address from localhost IP on all nodes
sudo vim /etc/mongod.conf
# node 1 # network interfaces net: port: 27017 bindIp: 10.10.5.2 # Listen to local interface only, comment to listen on all interfaces. # node 2 # network interfaces net: port: 27017 bindIp: 10.10.5.3 # Listen to local interface only, comment to listen on all interfaces. # node 3 # network interfaces net: port: 27017 bindIp: 10.10.5.4 # Listen to local interface only, comment to listen on all interfaces.
Configure the storage path to the one created earlier on all nodes
# Where and how to store data.
storage:
dbPath: /data/mongodb
journal:
enabled: true
Configuring MongoDB Replica Set
One of the MongoDB nodes run as the PRIMARY, and all other nodes will work as SECONDARY. Data is always to the PRIMARY node and the data sets are then replicated to all other SECONDARYnodes.
Edit the MongoDB configuration file mongod.conf and enable replica set on all nodes
replication: replSetName: "replica01"
Open port 27017/tcp on the firewall:
sudo ufw enable sudo ufw allow ssh sudo ufw allow 27017/tcp
Configure MongoDB to start during the operating system’s boot
sudo systemctl enable mongod.service sudo systemctl restart mongod.service
Check the listen Address of MongoDB service:
# ss -tunelp | grep -i mongo
tcp LISTEN 0 128 10.10.5.2:27017 0.0.0.0:* users:(("mongod",pid=15288,fd=11)) uid:111 ino:46927 sk:4 <->
Initiate MongoDB Replica Set
Our MongoDB Node1 (mongodb-01) will be the PRIMARY and the other two will act as SECONDARY
Login to the mongodb-01 server and start the mongo shell.
$ mongo 10.10.5.2 MongoDB shell version v4.0.1 connecting to: mongodb://10.10.5.2:27017/test MongoDB server version: 4.0.1 Welcome to the MongoDB shell. For interactive help, type "help". ... >
Initialize replica set on node1 by running below command:
> rs.initiate()
{
"info2" : "no configuration specified. Using a default configuration for the set",
"me" : "10.10.5.2:27017",
"ok" : 1,
"operationTime" : Timestamp(1534797235, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1534797235, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
Make sure you get 1 for ok state
Add secondary nodes
replica01:PRIMARY> rs.add("mongodb-02")
{
"ok" : 1,
"operationTime" : Timestamp(1534797580, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1534797580, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
replica01:PRIMARY> rs.add("mongodb-03")
{
"ok" : 1,
"operationTime" : Timestamp(1534797614, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1534797614, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
Check replica set status using:
replica01:PRIMARY> rs.status()
Check the status of the master
replica01:PRIMARY> rs.isMaster()
Testing MongoDB Replication
Create a test database on mongodb-01
replica01:PRIMARY> use test_db switched to db test_db
Add some data
> db.test.save(
{
"desc": "My Test Database",
"apps": ["Test1", "Test2", "Test3", "Test4"],
})
replica01:PRIMARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
test_db 0.000GB
replica01:PRIMARY> use test_db
switched to db test_db
replica01:PRIMARY> show collections
test
Conclusion
You now have a working MongoDB replication. You can increase the cluster size from three nodes to five nodes, seven nodes or more. This number is calculated from: [Master Eligible Nodes) / 2 + 1], which means an odd number of nodes is required for high availability, the minimum number is three.
Reference:
https://docs.mongodb.com/master/replication/
https://docs.mongodb.com/manual/tutorial/deploy-replica-set/
