[Hampshire] qemu and paravirtual devices

I've recently been playing with qemu's support for paravirtual
devices. Damian had expressed an interest (as we use qemu+kqemu VMs
heavily at work), but I thought this might be of interest to others as
well.

What are paravirtual devices?
-----------------------------

When running a virtual machine, the virtual environment has to
present devices to the guest OS -- disks and network being the main
two (plus video, USB, timers, and others). Effectively, this is the
hardware that the VM guest sees.

Now, if the guest is to be kept entirely ignorant of the fact that
it is virtualised, this means that the host must emulate some kind of
real hardware. This is quite slow (particularly for network devices),
and is the major cause of reduced performance in virtual machines.

However, if you are willing to let the guest OS know that it's in a
virtual environment, it is possible to avoid the overheads of
emulating much of the real hardware, and use a far more direct path to
handle devices inside the VM. This approach is called
paravirtualisation. In this case, the guest OS needs a particular
driver installed which talks to the paravirtual device. Under Linux,
this interface has been standardised, and is referred to as the
"virtio" interface.

What do you need to make this work?
-----------------------------------

- A patched version of qemu
- A guest kernel with paravirtual / virtio support
- A patched version of the LVM tools, if you want to use LVM in the
guest on the virtio block device.

Patching qemu
-------------

Grab the latest 0.9.1 qemu sources, and the virtio device patches.
The original patches are available from [1,2,3,4], but they don't
apply cleanly to the 0.9.1 source. There are updated patches at
[5,6,7] if you don't feel like porting the patches yourself.

Apply the patches, build and install qemu as normal.

Guest kernel
------------

In order to get the necessary support in the guest kernel, you will
need the latest pre-release kernel. You can get this either from
Linus's git repository, or (as of this weekend) as 2.6.25-rc1 or later
from the kernel.org repository[8]. If you are running Ubuntu in the
guest, I'm told that the Hardy Alpha 4 kernel has the necessary
patches in it.

The configuration options you need turned on are:

PARAVIRT_GUEST:
    -> Processor type and features
        -> Paravirtualized guest support

LGUEST_GUEST:
    -> Processor type and features
        -> Paravirtualized guest support
            -> Lguest guest support

VIRTIO_PCI:
    -> Virtualization (VIRTUALIZATION [=y])
        -> PCI driver for virtio devices

VIRTIO_BLK:
    -> Device Drivers
        -> Block devices (BLK_DEV [=y])
            -> Virtio block driver

VIRTIO_NET:
    -> Device Drivers
        -> Network device support (NETDEVICES [=y])
            -> Virtio network driver

I suggest the latter three devices are built as modules.

There is also a VIRTIO_BALLOON driver for dealing with dynamic
memory allocation, and a VIRTIO_CONSOLE driver. I don't know if either
of these has support in qemu yet -- I suspect not.

Starting qemu
-------------

To start qemu with a virtio block device, you will need at minimum
the following option:

-drive file=/dev/path/to/host/device,if=virtio

This will map the given block device or file on the host as a
virtio-blk device on the guest. You can specify more -drive options to
add more virtio devices.

Accessing your virtio device
----------------------------

In the guest OS, you will need the modules virtio-blk and
virtio-pci loaded. This should then create devices /dev/vda, /dev/vdb,
etc. -- one for each virtio -drive option you specified on the command
line to qemu. These devices can be treated just like any other hard
disk -- they can be partitioned, formatted, and filesystems mounted on
them. Note that at the moment, there doesn't seem to be any support
for booting off them, so you will need at least one non-virtio device
in the VM.

Logical volume manager
----------------------

In getting this working, I discovered that the LVM tools don't know
about virtio block devices, and thus ignore them. There is a simple
one-line patch at [9] which teaches the LVM tools about virtio
devices. With that patch included, it all works nicely.

Virtio network
--------------

To run a virtio-type network card, you will need the virtio-net and
virtio-pci modules loaded in the guest kernel. You can then create a
virtio network device in the guest system with the qemu option:

-net nic,model=virtio

Of course, you will also need a -net tap, -net user or -net socket
option to create the paired device on the host side, as normal. Your
network device will then appear as eth0. I haven't managed to get my
virtio network device working properly yet, so your mileage may vary
with this one. My problem is that it's simply not talking to the
outside world -- the device exists and can be manipulated with
ifconfig, and it responds to pings, but no data seems to reach the
host network.

Performance
-----------

I've not tested performance on the network side (because I can't
get it to work), but my tests on the block device show:

Kernel compile, using qemu's IDE emulation:

real    53m56.731s
user    23m38.001s
sys    26m33.772s

Kernel compile, using qemu's virtio devices:

real    33m21.242s
user    10m52.030s
sys    11m32.950s

I'd say that's a win.

Hugo.

[1] http://lists.gnu.org/archive/html/qemu-devel/2007-12/msg00093.html
[2] http://lists.gnu.org/archive/html/qemu-devel/2007-12/msg00094.html
[3] http://lists.gnu.org/archive/html/qemu-devel/2007-12/msg00095.html
[4] http://lists.gnu.org/archive/html/qemu-devel/2007-12/msg00096.html
[5] http://www.carfax.org.uk/virtio-1.patch
[6] http://www.carfax.org.uk/virtio-2.patch
[7] http://www.carfax.org.uk/virtio-3.patch
[8] http://www.eu.kernel.org/pub/linux/kernel/v2.6/testing/
[9] http://www.carfax.org.uk/lvm-virtio.patch

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Geek, n.: Circus sideshow performer specialising in the ---     
                         eating of live animals.