Raspberry Pi 2 uses bcm2709 which is an ARMV7 processor.  


The processor is CORTEX-A7 with a virtualization extension. However, this feature is not supported in the current raspbian system. This article is about how to implement virtualization for Raspberry Pi 2.

We use uboot+zImag+DTB to boot the machine here.

cat /proc/cpuinfo

Hardware    : BCM2709
Revision    : a01041

model name  : ARMv7 Processor rev 5 (v7l)
BogoMIPS    : 64.00
Features    : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xc07
CPU revision    : 5

First, enable HYP mode.

[    0.154021] CPU: All CPU(s) started in SVC mode

Rsapberry Pi 2 jumps to SVC mode instead of HYP mode.

Solution:

Build a bootloader that properly sets HYP mode before jumping to 0x8000.

So, build a new u-boot, and connect to zImage.

git clone https://github.com/slp/rpi2-hyp-boot.git
cd rpi2-hyp-boot
make

this will get a bootblk.bin file which contains boot code and 32K padding, then  

mv /boot/kernel7.img /boot/kernel7.img.bak
cat bootblk.bin /boot/kernel7.img.bak > /boot/kernel7.img
echo "kernel_old=1" >> /boot/config.txt

This will make raspberry pi get into HYP mode.

[    0.154131] CPU: All CPU(s) started in HYP mode.
[    0.154158] CPU: Virtualization extensions available.

However, it is not over yet, we need to run the KVM-support VM finally.

Second, modification for the host kernel.

make sure the following options are enabled:

  • Patch physical to virtual translations at runtime
  • General setup -> Control Group support
  • System Type -> Support for Large Physical Address Extension
  • Boot options -> Use appended device tree blob to zImage (EXPERIMENTAL)
  • Boot options -> Supplement the appended DTB with traditional ATAG information
  • Device Drivers -> Block devices -> Loopback device support
  • Virtualization
  • Virtualization -> Kernel-based Virtual Machine (KVM) support (NEW)
  • DISABLE Virtualization -> KVM support for Virtual GIC
  • ENABLE Virtualization -> KVM support for Emulated GIC

Then cross-compiler this kernel.

wget http://releases.linaro.org/14.04/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.bz2

tar -xvf  gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.bz2

Add the path of gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux/bin to $PATH.

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- zImage dtbs

Last, wrap all things to a new kernel, kernel7.img by the following steps:

cat rpi2-hyp-boot/bootblk.bin linux/arch/arm/boot/zImage linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb > kernel7.img

Edit /boot/cmdline.txt and add isolcpus=3 at the of the line

Third, modification on QEMU.

Rember to let QEMU support CPU affinity.

wget http://wiki.qemu-project.org/download/qemu-2.2.0.tar.bz2
patch -p1 < ~/qemu-cpu-affinity.patch

After all setup, we need to build a guest vm kernel and dtb, build a rootfs. use qemu to boot it.

path-to-qemu/arm-softmmu/qemu-system-arm \

-enable-kvm -smp 1 -m 256 -M vexpress-a15 -cpu host \
-kernel vexpress-zImage \
-dtb vexpress-v2p-ca15-tc1.dtb \
-append "root=/dev/vda console=ttyAMA0 rootwait" \
-drive if=none,file=opensuse-factory.img,id=factory \
-device virtio-blk-device,drive=factory \
-monitor null -serial stdio \
-nographic

The only tricky part is to build roofs img files.

With all three steps above we could run a KVM-support machine successfully.

Reference :

1. https://www.raspberrypi.org/documentation/linux/kernel/building.md

2. http://blog.flexvdi.com/2015/03/17/enabling-kvm-virtualization-on-the-raspberry-pi-2/

3. https://lwn.net/Articles/557132/