(Last Updated On: May 5, 2019)

If you’re a fan of terraform and KVM, I’m assured you’ve been looking for a way to provision Virtual Machines on KVM in automated manner with Terraform. In this blog post, I’ll walk you through installation of Terraform KVM provider and using it to manage instances running on KVM hypervisor.

Terraform is an open-source infrastructure as code software tool created by HashiCorp. It allows you to safely and predictably create, change, and improve infrastructure. All your infrastructure code can be saved in a Git repository and versioned.

A provider in Terraform is responsible for the lifecycle of a resource: create, read, update, delete. Hashicorp has a number of officially supported providers available for use. Unfortunately, KVM is not in the list.

Step 1: Install KVM hypervisor

The major pre-requisite for this setup is KVM hypervisor. Install KVM in your Linux system by referring to a relevant article from the list below.

How to install KVM on RHEL / CentOS 8

How to install KVM on Fedora

Install KVM on CentOS 7 / Ubuntu / Debian / SLES

Install KVM on Arch Linux / Manjaro

The KVM service (libvird) should be running and enabled to start at boot.

sudo systemctl start libvirtd
sudo systemctl enable libvirtd

Enable vhost-net kernel module on Ubuntu/Debian.

sudo modprobe vhost_net
echo vhost_net | sudo tee -a /etc/modules

If you want to generate KVM VM templates, refer to How to Create CentOS / Fedora / RHEL VM Templates on KVM

Step 2: Install Terraform

After installing and starting KVM, do Terraform installation.

How to Install Terraform on Ubuntu / Debian / CentOS / Arch / Fedora

Terraform installation is much easier. You just need to downloaded a binary archive, extract and place the binary file in a directory in your $PATH.

Step 3: Install Terraform KVM provider

The Terraform KVM provider will provision infrastructure with Linux’s KVM using libvirt. It is maintained by Duncan Mac-Vicar P with other contributors.

Initialize Terraform working directory.

$ cd ~
$ terraform init
Terraform initialized in an empty directory!

Create directory for storing Terraform Plugins.

cd ~/.terraform.d
mkdir plugins

Check the Github releases page for available downloads.

Install Terraform KVM provider on Ubuntu / Linux Mint

sudo apt -y install wget
wget https://github.com/dmacvicar/terraform-provider-libvirt/releases/download/v0.5.1/terraform-provider-libvirt-0.5.1.Ubuntu_18.04.amd64.tar.gz
tar xvf terraform-provider-libvirt-0.5.1.Ubuntu_18.04.amd64.tar.gz

Move terraform-provider-libvirt binary file to the ~/.terraform.d/plugins directory.

mv terraform-provider-libvirt ~/.terraform.d/plugins/

Install Terraform KVM provider on CentOS 7 / Fedora / OpenSUSE

Run commands below if you are running Fedora or CentOS in your Workstation.

# CentOS 7
sudo yum -y install wget
wget https://github.com/dmacvicar/terraform-provider-libvirt/releases/download/v0.5.1/terraform-provider-libvirt-0.5.1.CentOS_7.x86_64.tar.gz
tar xvf terraform-provider-libvirt-0.5.1.CentOS_7.x86_64.tar.gz
mv terraform-provider-libvirt ~/.terraform.d/plugins/

# Fedora
sudo dnf -y install wget
wget xvf https://github.com/dmacvicar/terraform-provider-libvirt/releases/download/v0.5.1/terraform-provider-libvirt-0.5.1.Fedora_28.x86_64.tar.gz
tar xvf terraform-provider-libvirt-0.5.1.Fedora_28.x86_64.tar.gz
mv terraform-provider-libvirt ~/.terraform.d/plugins/

Install Terraform KVM provider from source

For other systems, you can build Terraform libvirt provider from the source. For this you need.

  • libvirt 1.2.14 or newer on the hypervisor
  • The latest version of golang
  • mkisofs is required to use the CloudInit feature.
  • cgo is required by the libvirt-go package. export CGO_ENABLED=”1″

Install Go on your Linux system with the help of:

How to Install Go (Golang) on Linux Mint 19

How to Install Go on RHEL 8

How to Install Go on Ubuntu 18.04/ CentOS 7

Go can be installed on Arch Linux / Manjaro by running:

sudo pacman -S go cdrkit

Ensure you have $GOPATH configured in your Linux system. Then proceed to download and install the provider.

go get github.com/dmacvicar/terraform-provider-libvirt
go install github.com/dmacvicar/terraform-provider-libvirt

The binary file will be located at $GOPATH/bin/terraform-provider-libvirt. Copy it to the plugins directory.

cp $GOPATH/bin/terraform-provider-libvirt ~/.terraform.d/plugins

Using Terraform KVM Provider

Once you have provider inside plugins directory. Create your Terraform projects folder.

mkdir ~/projects/terraform
cd ~/projects/terraform

Create libvirt.tf file for your VM deployment on KVM.

provider "libvirt" {
  uri = "qemu:///system"
}

#provider "libvirt" {
#  alias = "server2"
#  uri   = "qemu+ssh://[email protected]/system"
#}

resource "libvirt_volume" "centos7-qcow2" {
  name = "centos7.qcow2"
  pool = "default"
  source = "https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2"
  #source = "./CentOS-7-x86_64-GenericCloud.qcow2"
  format = "qcow2"
}

# Define KVM domain to create
resource "libvirt_domain" "db1" {
  name   = "db1"
  memory = "1024"
  vcpu   = 1

  network_interface {
    network_name = "default"
  }

  disk {
    volume_id = "${libvirt_volume.centos7-qcow2.id}"
  }

  console {
    type = "pty"
    target_type = "serial"
    target_port = "0"
  }

  graphics {
    type = "spice"
    listen_type = "address"
    autoport = true
  }
}

Initialize a Terraform working directory:

$ terraform init
Initializing provider plugins…

Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Generate and show Terraform execution plan

$ terraform plan

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + libvirt_domain.db1
      id:                               <computed>
      arch:                             <computed>
      console.#:                        "1"
      console.0.target_port:            "0"
      console.0.target_type:            "serial"
      console.0.type:                   "pty"
      disk.#:                           "1"
      disk.0.scsi:                      "false"
      disk.0.volume_id:                 "${libvirt_volume.centos7-qcow2.id}"
      emulator:                         <computed>
      graphics.#:                       "1"
      graphics.0.autoport:              "true"
      graphics.0.listen_address:        "127.0.0.1"
      graphics.0.listen_type:           "address"
      graphics.0.type:                  "spice"
      machine:                          <computed>
      memory:                           "1024"
      name:                             "db1"
      network_interface.#:              "1"
      network_interface.0.addresses.#:  <computed>
      network_interface.0.hostname:     <computed>
      network_interface.0.mac:          <computed>
      network_interface.0.network_id:   <computed>
      network_interface.0.network_name: "default"
      qemu_agent:                       "false"
      running:                          "true"
      vcpu:                             "1"

  + libvirt_volume.centos7-qcow2
      id:                               <computed>
      format:                           "qcow2"
      name:                             "centos7.qcow2"
      pool:                             "default"
      size:                             <computed>
      source:                           "./CentOS-7-x86_64-GenericCloud.qcow2"


Plan: 2 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Then build your Terraform infrastructure if desired state is confirmed to be correct.

$ terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + libvirt_domain.db1
      id:                               <computed>
      arch:                             <computed>
      console.#:                        "1"
      console.0.target_port:            "0"
      console.0.target_type:            "serial"
      console.0.type:                   "pty"
      disk.#:                           "1"
      disk.0.scsi:                      "false"
      disk.0.volume_id:                 "${libvirt_volume.centos7-qcow2.id}"
      emulator:                         <computed>
      graphics.#:                       "1"
      graphics.0.autoport:              "true"
      graphics.0.listen_address:        "127.0.0.1"
      graphics.0.listen_type:           "address"
      graphics.0.type:                  "spice"
      machine:                          <computed>
      memory:                           "1024"
      name:                             "db1"
      network_interface.#:              "1"
      network_interface.0.addresses.#:  <computed>
      network_interface.0.hostname:     <computed>
      network_interface.0.mac:          <computed>
      network_interface.0.network_id:   <computed>
      network_interface.0.network_name: "default"
      qemu_agent:                       "false"
      running:                          "true"
      vcpu:                             "1"

  + libvirt_volume.centos7-qcow2
      id:                               <computed>
      format:                           "qcow2"
      name:                             "centos7.qcow2"
      pool:                             "default"
      size:                             <computed>
      source:                           "./CentOS-7-x86_64-GenericCloud.qcow2"


Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Press “yes” to confirm execution. Below is my terraform execution output.

libvirt_volume.centos7-qcow2: Creating...
  format: "" => "qcow2"
  name:   "" => "db.qcow2"
  pool:   "" => "default"
  size:   "" => "<computed>"
  source: "" => "./CentOS-7-x86_64-GenericCloud.qcow2"
libvirt_volume.centos7-qcow2: Creation complete after 8s (ID: /var/lib/libvirt/images/db.qcow2)
libvirt_domain.db1: Creating...
  arch:                             "" => "<computed>"
  console.#:                        "" => "1"
  console.0.target_port:            "" => "0"
  console.0.target_type:            "" => "serial"
  console.0.type:                   "" => "pty"
  disk.#:                           "" => "1"
  disk.0.scsi:                      "" => "false"
  disk.0.volume_id:                 "" => "/var/lib/libvirt/images/db.qcow2"
  emulator:                         "" => "<computed>"
  graphics.#:                       "" => "1"
  graphics.0.autoport:              "" => "true"
  graphics.0.listen_address:        "" => "127.0.0.1"
  graphics.0.listen_type:           "" => "address"
  graphics.0.type:                  "" => "spice"
  machine:                          "" => "<computed>"
  memory:                           "" => "1024"
  name:                             "" => "db1"
  network_interface.#:              "" => "1"
  network_interface.0.addresses.#:  "" => "<computed>"
  network_interface.0.hostname:     "" => "<computed>"
  network_interface.0.mac:          "" => "<computed>"
  network_interface.0.network_id:   "" => "<computed>"
  network_interface.0.network_name: "" => "default"
  qemu_agent:                       "" => "false"
  running:                          "" => "true"
  vcpu:                             "" => "1"
libvirt_domain.db1: Creation complete after 0s (ID: e5ee28b9-e1da-4945-9eb0-0cda95255937)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Confirm VM creation with virsh command.

$ sudo virsh  list
 Id   Name   State
----------------------
 7    db1    running

Get Instance IP address.

$ sudo virsh net-dhcp-leases default 
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
------------------------------------------------------------------------------------------------------------------------------------------------
 2019-03-24 16:11:18   52:54:00:3e:15:9e   ipv4       192.168.122.61/24    -          -
 2019-03-24 15:30:18   52:54:00:8f:8c:86   ipv4       192.168.122.198/24   rhel8      ff:61:69:21:bd:00:02:00:00:ab:11:0e:9c:c6:63:ee:7d:c8:d1

My instance IP is 192.168.122.61. I can ping the instance.

$  ping -c 1 192.168.122.61 
PING 192.168.122.61 (192.168.122.61) 56(84) bytes of data.
64 bytes from 192.168.122.61: icmp_seq=1 ttl=64 time=0.517 ms

--- 192.168.122.61 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.517/0.517/0.517/0.000 ms

To destroy your infrastructure, run:

terraform destroy

Using cloud-init with Terraform Libvirt provider

The instance resource we used didn’t have an option for passing user password. So if you’re using cloud template which doesn’t support password authentication, you won’t be able to login. Luckily, we can use libvirt_cloudinit_disk resource to pass user data to the instance.

Create Cloud init configuration file.

$ cat cloud_init.cfg
#cloud-config
# vim: syntax=yaml
#
# ***********************
# 	---- for more examples look at: ------
# ---> https://cloudinit.readthedocs.io/en/latest/topics/examples.html
# ******************************
#
# This is the configuration syntax that the write_files module
# will know how to understand. encoding can be given b64 or gzip or (gz+b64).
# The content will be decoded accordingly and then written to the path that is
# provided.
#
# Note: Content strings here are truncated for example purposes.
ssh_pwauth: True
chpasswd:
  list: |
     root: StrongPassword
  expire: False

users:
  - name: jmutai # Change me
    ssh_authorized_keys:
      - ssh-rsa AAAAXX #Chageme
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    shell: /bin/bash
    groups: wheel
  • This will set root password to StrongPassword
  • Add user named jmutai with specified Public SSH keys
  • The user will be added to wheel group and be allowed to run sudo commands without password.

Edit libvirt.tf to use Cloud init configuration file.

provider "libvirt" {
  uri = "qemu:///system"
}

#provider "libvirt" {
#  alias = "server2"
#  uri   = "qemu+ssh://[email protected]/system"
#}

resource "libvirt_volume" "centos7-qcow2" {
  name = "db.qcow2"
  pool = "default"
  #source = "https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2"
  source = "./CentOS-7-x86_64-GenericCloud.qcow2"
  format = "qcow2"
}

data "template_file" "user_data" {
  template = "${file("${path.module}/cloud_init.cfg")}"
}

# Use CloudInit to add the instance
resource "libvirt_cloudinit_disk" "commoninit" {
  name = "commoninit.iso"
  user_data      = "${data.template_file.user_data.rendered}"
}

# Define KVM domain to create
resource "libvirt_domain" "db1" {
  name   = "db1"
  memory = "1024"
  vcpu   = 1

  network_interface {
    network_name = "default"
  }

  disk {
    volume_id = "${libvirt_volume.centos7-qcow2.id}"
  }

  cloudinit = "${libvirt_cloudinit_disk.commoninit.id}"

  console {
    type = "pty"
    target_type = "serial"
    target_port = "0"
  }

  graphics {
    type = "spice"
    listen_type = "address"
    autoport = true
  }
}

# Output Server IP
output "ip" {
  value = "${libvirt_domain.db1.network_interface.0.addresses.0}"
}

Re-initialize Terraform working directory.

$ terraform init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "template" (2.1.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.template: version = "~> 2.1"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Then run

terraform plan
terraform apply

Execution output:

Note the Server IP printed on the screen.

Or use virsh command to get the server IP address.

$ sudo virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
---------------------------------------------------------------------------------------------------------
 2019-03-24 16:41:32   52:54:00:22:45:57   ipv4       192.168.122.219/24   -          -

Try login to the instance as root user and password set.

Check if ssh user created can login with SSH key and run sudo without password.

$ ssh [email protected]
The authenticity of host '192.168.122.219 (192.168.122.219)' can't be established.
ECDSA key fingerprint is SHA256:G8ByhT4+FXBzh/MabB67rcS6JpTUn1TcrusXhiy8ke0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.122.219' (ECDSA) to the list of known hosts.

[[email protected] ~]$ sudo su -
Last login: Sun Mar 24 13:16:46 UTC 2019 on tty1
[[email protected] ~]#

Check Terraform KVM provider documentation for resources usage and provided examples on how to use the provider. Refer to Cloud Init Documentation for its usage.

Install Terraform on Windows 10 / Windows Server 2019

Install and Configure Hashicorp Vault Server on Ubuntu / CentOS / Debian

How to run Minikube on KVM

virsh commands cheatsheet to manage KVM guest virtual machines