Run a kvm-support VM on Raspberry Pi 2

lyan 2015-10-30 02:56

Raspberry Pi 2 is powered by BCM2709, a quad-core Cortex-A7 (ARMv7-A) processor. Interestingly, Cortex-A7 includes the ARM virtualization extensions.

In theory, this means Raspberry Pi 2 should support hardware-assisted virtualization. In practice, it does not — at least not out of the box.

This article documents the full process of enabling KVM on Raspberry Pi 2, including bootloader modification, kernel rebuild, and QEMU adjustments.

Step 0: The First Discovery — We Are Not in HYP Mode

After booting a stock Raspbian system, the kernel log shows:

[    0.154021] CPU: All CPU(s) started in SVC mode

This is the key problem.

ARM virtualization requires the processor to operate in HYP mode (the ARMv7 hypervisor privilege level). However, Raspberry Pi 2 boots directly into SVC mode.

If the system does not start in HYP mode, KVM cannot initialize — regardless of kernel configuration.

Step 1: Forcing the System into HYP Mode

ARMv7 does not allow transitioning into HYP mode after normal kernel boot. The processor must enter HYP mode before the kernel fully initializes.

Therefore, the solution is not a kernel patch — it is a bootloader modification.

Using rpi2-hyp-boot

git clone https://github.com/slp/rpi2-hyp-boot.git
cd rpi2-hyp-boot
make

This produces bootblk.bin, which contains code that switches the CPU into HYP mode before jumping to the kernel entry point (0x8000).

Rebuilding kernel7.img

mv /boot/kernel7.img /boot/kernel7.img.bak
cat bootblk.bin /boot/kernel7.img.bak > /boot/kernel7.img
echo "kernel_old=1" >> /boot/config.txt

After reboot:

[    0.154131] CPU: All CPU(s) started in HYP mode.
[    0.154158] CPU: Virtualization extensions available.

Now the processor is running in HYP mode. This is the fundamental requirement for ARM KVM.

Step 2: Rebuilding the Host Kernel with KVM Support

Entering HYP mode alone is not sufficient. The kernel must also support:

LPAE (Large Physical Address Extension)
KVM
Emulated GIC

LPAE is mandatory because ARM virtualization relies on Stage-2 address translation. Without LPAE, Stage-2 page tables cannot function.

Cross Compilation Toolchain

wget http://releases.linaro.org/14.04/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.bz2
tar -xvf gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.bz2
export PATH=$PATH:/path/to/linaro/bin

Build zImage and DTBs

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- zImage dtbs

Reconstruct kernel7.img

cat bootblk.bin \
    arch/arm/boot/zImage \
    arch/arm/boot/dts/bcm2709-rpi-2-b.dtb \
    > kernel7.img

Optional: isolate one CPU core for host stability.

isolcpus=3

Step 3: Adjusting QEMU for KVM Execution

With the host prepared, the final step is enabling KVM acceleration in QEMU. In some cases, patching QEMU for proper CPU affinity support improves stability.

wget http://wiki.qemu-project.org/download/qemu-2.2.0.tar.bz2
patch -p1 < ~/qemu-cpu-affinity.patch

Booting a KVM-Enabled Guest

Once the guest kernel, DTB, and rootfs are prepared, the VM can be launched with:

qemu-system-arm \
-enable-kvm \
-smp 1 -m 256 \
-M vexpress-a15 \
-cpu host \
-kernel vexpress-zImage \
-dtb vexpress-v2p-ca15-tc1.dtb \
-append "root=/dev/vda console=ttyAMA0 rootwait" \
-drive if=none,file=opensuse-factory.img,id=factory \
-device virtio-blk-device,drive=factory \
-nographic

If everything is configured correctly, the guest will boot using hardware-assisted virtualization.

Why This Matters

This exercise demonstrates several key architectural realities:

ARM virtualization depends on early privilege level configuration.
HYP mode must be enabled before kernel execution.
LPAE is required for Stage-2 memory translation.
Interrupt virtualization (GIC configuration) is critical for stability.

Unlike x86, where virtualization is often transparent, ARM virtualization exposes architectural boundaries between:

Bootloader
Kernel privilege levels
Memory translation stages
Interrupt controller design

Running KVM on Raspberry Pi 2 is not just about launching a VM. It is about understanding how ARM hardware virtualization actually works.

Small board. Full hypervisor stack. Complete control.