(Last Updated On: July 4, 2018)

In this guide, we’ll look at how to use Vagrant with Libvirt on Linux. Vagrant is an open-source software written in Ruby to help you build and maintain a portable virtual software development environments, e.g. for VirtualBox, Hyper-V, Docker containers, VMware, Libvirt, and AWS. Vagrant makes it easy to create, stop and destroy Virtual Machines.

The most Vagrant use case on Local machines is the use of VirtualBox provider. If you are a KVM and QEMU user, this article is for you. KVM has been known to have a better performance and small resource utilization as compared to Virtualbox.

Vagrant with Libvirt on Linux Prerequisites:

  1. Installed Vagrant
  2. Installed Libvirt and QEMU-KVM
  3. Installing libvirt plugin for Vagrant

For installation of KVM on Linux, we have some articles already baked for you.

Install KVM on CentOS 7 / Ubuntu 16.04 / Debian 9 / SLES 12 / Arch Linux

Complete Installation of KVM,QEMU and Virt Manager on Arch Linux and Manjaro

At least mostly used, Linux Distributions have been covered. For installation of Vagrant, please consult your Vagrant official documentation.

Installing Vagrant plugin for Vagrant

Once you have Vagrant and KVM installed, you should be ready to install a libvirt plugin so that you can start managing KVM Virtual machines using Vagrant.

$ vagrant plugin install vagrant-libvirt
Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Building native extensions. This could take a while...
Building native extensions. This could take a while...
Installed the plugin 'vagrant-libvirt (0.0.43)'!

If you encounter an error like below:

ERROR: Failed to build gem native extension.

current directory: /home/jmutai/.vagrant.d/gems/2.5.1/gems/nokogiri-1.8.4/ext/nokogiri
/usr/bin/ruby -r ./siteconf20180704-25314-14hvlbq.rb extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using system libraries.
pkg-config could not be used to find libxml-2.0
Please install either `pkg-config` or the pkg-config gem per
gem install pkg-config -v "~> 1.1"

Then run:

$ gem install nokogiri
$ vagrant plugin install pkg-config

And retry installing the plugin.

$ vagrant plugin install vagrant-libvirt 
Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Fetching: excon-0.62.0.gem (100%)
Fetching: formatador-0.2.5.gem (100%)
Fetching: fog-core-1.43.0.gem (100%)
Fetching: fog-json-1.2.0.gem (100%)
Fetching: mini_portile2-2.3.0.gem (100%)
Building native extensions. This could take a while...
Fetching: fog-xml-0.1.3.gem (100%)
Fetching: ruby-libvirt-0.7.1.gem (100%)
Building native extensions. This could take a while...
Fetching: fog-libvirt-0.5.0.gem (100%)
Fetching: vagrant-libvirt-0.0.43.gem (100%)
Installed the plugin 'vagrant-libvirt (0.0.43)'!

Once the installation is complete, you can confirm that the plugin has been installed using the following command:

$ vagrant plugin list
vagrant-libvirt (0.0.43)

Downloading Vagrant boxes

A Vagrant box for Libvirt is a tar archive with 3 files in it.

  • A base VagrantFile
  • Th metadata.json file
  • QCOW2 image

If you are interested in building your own Vagrant boxes, have a look at using Packer and Packer build templates for Vagrant on chef/bento github repo. It will help you to easily get started. In this example, we’re going to use a ready template. Let add CentOS 7 and CentOS 6 boxes.

$ vagrant box add centos/7 --provider=libvirt
==> box: Loading metadata for box 'centos/7'
 box: URL: https://vagrantcloud.com/centos/7
==> box: Adding box 'centos/7' (v1803.01) for provider: libvirt
$ vagrant box add centos/6 --provider=libvirt

Check the list of boxes presents locally.

$ vagrant box list
centos/6 (libvirt, 1803.01)
centos/7 (libvirt, 1803.01)
generic/ubuntu1604 (libvirt, 1.5.0)

Create VM Vagrantfile

Vagrant needs a configuration file to get the details and settings for a VM to be created. Let’s create a single VM Vagrantfile.

$ mkdir ~/vagrant-vms
$ cd ~/vagrant-vms

Create a Vagrantfile with content similar to below:

# -*- mode: ruby -*-
# vi: set ft=ruby :

ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'

# Check required plugins
REQUIRED_PLUGINS_LIBVIRT = %w(vagrant-libvirt)
exit unless REQUIRED_PLUGINS_LIBVIRT.all? do |plugin|
  Vagrant.has_plugin?(plugin) || (
    puts "The #{plugin} plugin is required. Please install it with:"
    puts "$ vagrant plugin install #{plugin}"
    false
  )
end

Vagrant.configure("2") do |config|

  # Rabbitmq VM
  config.vm.define "rabbitmq-centos6" do |node|
    node.vm.hostname = "rabbitmq-server-01"
    node.vm.box = "centos/6"
    node.vm.box_check_update = false
    #node.vm.synced_folder '.', '/vagrant', :disabled => true
    node.vm.network "private_network", ip: "192.168.18.15"
    node.vm.provider :libvirt do |domain|
      domain.memory = 512
      domain.nested = true
    end
  end
end

To bring the VM up, run:

$ vagrant up
Bringing machine 'rabbitmq-centos6' up with 'libvirt' provider...
==> rabbitmq-centos6: Uploading base box image as volume into libvirt storage...
==> rabbitmq-centos6: Creating image (snapshot of base box volume).
==> rabbitmq-centos6: Creating domain with the following settings...
==> rabbitmq-centos6:  -- Name:              rabbitmq-server_rabbitmq-centos6
==> rabbitmq-centos6:  -- Domain type:       kvm
==> rabbitmq-centos6:  -- Cpus:              1
==> rabbitmq-centos6: 
==> rabbitmq-centos6:  -- Feature:           acpi
==> rabbitmq-centos6:  -- Feature:           apic
==> rabbitmq-centos6:  -- Feature:           pae
==> rabbitmq-centos6:  -- Memory:            512M
==> rabbitmq-centos6:  -- Management MAC:    
==> rabbitmq-centos6:  -- Loader:            
==> rabbitmq-centos6:  -- Base box:          centos/6
==> rabbitmq-centos6:  -- Storage pool:      default
==> rabbitmq-centos6:  -- Image:             /var/lib/libvirt/images/rabbitmq-server_rabbitmq-centos6.img (41G)
==> rabbitmq-centos6:  -- Volume Cache:      default
==> rabbitmq-centos6:  -- Kernel:            
==> rabbitmq-centos6:  -- Initrd:            
==> rabbitmq-centos6:  -- Graphics Type:     vnc
==> rabbitmq-centos6:  -- Graphics Port:     -1
==> rabbitmq-centos6:  -- Graphics IP:       127.0.0.1
==> rabbitmq-centos6:  -- Graphics Password: Not defined
==> rabbitmq-centos6:  -- Video Type:        cirrus
==> rabbitmq-centos6:  -- Video VRAM:        9216
==> rabbitmq-centos6:  -- Sound Type:	
==> rabbitmq-centos6:  -- Keymap:            en-us
==> rabbitmq-centos6:  -- TPM Path:          
==> rabbitmq-centos6:  -- INPUT:             type=mouse, bus=ps2
==> rabbitmq-centos6: Creating shared folders metadata...
==> rabbitmq-centos6: Starting domain.
==> rabbitmq-centos6: Waiting for domain to get an IP address...
==> rabbitmq-centos6: Waiting for SSH to become available...
    rabbitmq-centos6: 
    rabbitmq-centos6: Vagrant insecure key detected. Vagrant will automatically replace
    rabbitmq-centos6: this with a newly generated keypair for better security.
    rabbitmq-centos6: 
    rabbitmq-centos6: Inserting generated public key within guest...
    rabbitmq-centos6: Removing insecure key from the guest if it's present...
    rabbitmq-centos6: Key inserted! Disconnecting and reconnecting using new SSH key...
==> rabbitmq-centos6: Setting hostname...
==> rabbitmq-centos6: Configuring and enabling network interfaces...
    rabbitmq-centos6: SSH address: 192.168.121.70:22
    rabbitmq-centos6: SSH username: vagrant
    rabbitmq-centos6: SSH auth method: private key

Run virsh list to see if you’ll get a list of VMs.

$ virsh list
 Id    Name                           State
----------------------------------------------------
 1     rabbitmq-server_rabbitmq-centos6 running

As defined in Vagrantfile, a new bridge will be created with a .1 IP Address from subnet defined, default mask is /24.

$ ip ad show dev virbr3
6: virbr3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:a6:63:05 brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.1/24 brd 192.168.18.255 scope global virbr3
       valid_lft forever preferred_lft forever

You can as well see the bridge using the brctl command:

$ brctl show
bridge name	bridge id		STP enabled	interfaces
virbr2		8000.5254003b9cd3	yes		virbr2-nic
							vnet0
virbr3		8000.525400a66305	yes		virbr3-nic
							vnet1

To ssh to the VM, use vagrant ssh command.

$ vagrant ssh        
Last login: Sat May 12 08:03:23 2018 from 192.168.121.1
[vagrant@rabbitmq-server-01 ~]$ 
[vagrant@rabbitmq-server-01 ~]$ ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:79:5b:09 brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.70/24 brd 192.168.121.255 scope global eth0
    inet6 fe80::5054:ff:fe79:5b09/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:1f:c1:de brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.15/24 brd 192.168.18.255 scope global eth1
    inet6 fe80::5054:ff:fe1f:c1de/64 scope link 
       valid_lft forever preferred_lft forever

To output .ssh/config valid syntax for connecting to this environment via ssh, run ssh-config command. You’ll need to place provided output under ~/.ssh/config directory to ssh.

$ vagrant ssh-config
Host rabbitmq-server-01
  HostName 192.168.121.122
  User vagrant
  Port 22
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /home/jmutai/hacks/vagrant/labs/rabbitmq-server/.vagrant/machines/rabbitmq-server-01/libvirt/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Then use ssh command to log in with name configured above:

$ ssh rabbitmq-server-01
Last login: Sat May 12 09:20:05 2018 from 192.168.121.1
[vagrant@rabbitmq-server-01 ~]$

To shut down the VM, run:

$ vagrant halt
==> rabbitmq-centos6: Halting domain...

$ virsh list --all
----------------------------------------------------
- centos_mysql-server-01 shut off
- rabbitmq-server_rabbitmq-centos6 shut off
- ubuntu-terraform shut off

To set VM to its initial state by cleaning all data, use vagrant destroy:

$ vagrant destroy
    rabbitmq-centos6: Are you sure you want to destroy the 'rabbitmq-centos6' VM? [y/N] y
==> rabbitmq-centos6: Removing domain...

$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     centos_mysql-server-01         shut off
 -     ubuntu-terraform               shut off

Build your own Vagrant box

You need packer installed for this to work.

$ wget https://releases.hashicorp.com/packer/1.2.3/packer_1.2.3_linux_amd64.zip
$ unzip packer_1.2.3_linux_amd64.zip 
Archive: packer_1.2.3_linux_amd64.zip
 inflating: packer
$ sudo cp packer /usr/local/bin 
$ type packer
packer is /usr/local/bin/packer

Then clone bento Github repo.

$ cd ~/
$ git clone https://github.com/chef/bento
$ cd bento
$ cd centos
$ packer build -only qemu -var "headless=true" centos-7.5-x86_64.json
==> qemu: Gracefully halting virtual machine...
==> qemu: Converting hard drive...
==> qemu: Running post-processor: vagrant
==> qemu (vagrant): Creating Vagrant box for 'libvirt' provider
 qemu (vagrant): Copying from artifact: ../builds/packer-centos-7.5-x86_64-qemu/centos-7.5-x86_64
 qemu (vagrant): Compressing: Vagrantfile
 qemu (vagrant): Compressing: box.img
 qemu (vagrant): Compressing: metadata.json
Build 'qemu' finished.
==> Builds finished. The artifacts of successful builds are:
--> qemu: 'libvirt' provider box: ../builds/centos-7.5.libvirt.box

If the build is successful, ready to import box files will be in the builds directory at the root of the repository.

$ vagrant box add builds/centos-7.5.libvirt.box --name "centos-7.5"
==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'centos-7.5' (v0) for provider: 
 box: Unpacking necessary files from: file:///home/jmutai/hacks/vagrant/labs/packer/bento/builds/centos-7.5.libvirt.box
==> box: Successfully added box 'centos-7.5' (v0) for 'libvirt'!

Verify the box is installed

$ vagrant box list
centos-7.5 (libvirt, 0)
centos/6 (libvirt, 1803.01)
centos/7 (libvirt, 1803.01)