Difference between revisions of "Xen ARM with Virtualization Extensions/qemu-system-aarch64"

From Xen
 
(7 intermediate revisions by 3 users not shown)
Line 7: Line 7:
 
== Build QEMU ==
 
== Build QEMU ==
   
  +
QEMU v4.0.0 can run Xen out of the box. Build QEMU like this:
QEMU v2.10 requires a few fixes to be able to run Xen on ARM in a nested fashion. QEMU v2.11 will likely work out of the box. For now, the simplest thing to do is to apply the appended patch. The following steps apply the patch and build QEMU:
 
   
  +
$ ./configure --target-list=aarch64-softmmu --prefix=/usr/local
$ git clone git://git.qemu.org/qemu.git
 
  +
$ make
 
  +
$ sudo make install
If you are running QEMU <= v2.10, save [https://wiki.xenproject.org/wiki/Qemu-aarch64.patch this patch] to a text file, then:
 
   
  +
== Install Xen ==
$ patch -p1 < /path/to/Qemu-aarch64.patch
 
   
  +
QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like [https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-uefi1.img xenial-server-cloudimg-arm64-uefi1.img].
Finally build QEMU:
 
   
  +
Then, clone Linux and build the kernel using the defconfig for arm64 (make defconfig && make Image.gz).
$ ./configure --target-list=aarch64-softmmu
 
  +
For building Linux kernel for ARM64 / ARM user might need cross compilers. Below are details with respect to different distros :
$ make
 
   
  +
opensuse : https://opensuse.pkgs.org/tumbleweed/opensuse-oss/cross-aarch64-gcc7-7.3.1+r258313-1.2.x86_64.rpm.html
== Install Xen ==
 
   
  +
ubuntu : https://packages.ubuntu.com/en/trusty/gcc-aarch64-linux-gnu
QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like [https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-uefi1.img xenial-server-cloudimg-arm64-uefi1.img], then:
 
   
  +
Other distros : https://pkgs.org/
$ losetup /dev/loop0 /path/to/xenial-server-cloudimg-arm64-uefi1.img
 
$ kpartx -a /dev/loop0
 
$ mount /dev/mapper/loop0p15 /mnt
 
   
  +
Steps can be found at http://events17.linuxfoundation.org/sites/events/files/slides/Shuah_Khan_cross_compile_linux.pdf
Follow [[Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM]] to build Xen on ARM. Then, copy the Xen on ARM efi binary to the image:
 
   
  +
Run QEMU the first time, booting Linux directly (replace the MAC_ADDRESS and the paths). The following command uses user-networking and forwards port 2222 from the host to port 22 inside the virtual machine:
$ cp /path/to/xen.git/xen/xen.efi /mnt
 
  +
  +
$ qemu-system-aarch64 \
  +
-machine virt,gic_version=3 -machine virtualization=true \
  +
-cpu cortex-a57 -machine type=virt -nographic \
  +
-smp 4 -m 4000 \
  +
-kernel /path/to/linux.git/arch/arm64/boot/Image.gz --append "console=ttyAMA0 root=/dev/vda1 init=/bin/sh" \
  +
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
  +
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0
  +
  +
This command will give a shell prompt, where user might have to mount the root file system using below command :
  +
  +
$ mount -o remount,rw /dev/vda1 /
  +
  +
Once mounted, user would like to set the 'root' password using 'passwd' command and then relaunch the virtual machine using above qemu command, but now remove 'init=/bin/sh'.
  +
  +
Once login with 'root' user, you may need to set the ssh authorized_keys of host machine, to access the virtual machine via remote.
  +
  +
Follow [[Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM]] to build Xen on ARM. Then, copy the Xen on ARM efi binary and the Linux kernel to the image:
  +
  +
$ scp -P2222 /path/to/linux.git/arch/arm64/boot/Image.gz root@127.0.0.1:/boot/efi/kernel
  +
$ scp -P2222 /path/to/xen.git/xen/xen.efi root@127.0.0.1:/boot/efi
   
 
Let's write a config file for it:
 
Let's write a config file for it:
   
$ vi /mnt/boot/xen.cfg
+
$ vi /boot/efi/xen.cfg
   
 
Let's use the following config:
 
Let's use the following config:
Line 42: Line 61:
 
dtb=virt-gicv3.dtb
 
dtb=virt-gicv3.dtb
   
  +
Now we need the device tree binary, "virt-gicv3.dtb". QEMU can generate one for you with the following command:
Build a Linux kernel image for Dom0 and copy the binary to /mnt/kernel.
 
   
  +
$ qemu-system-aarch64 \
Now we need the device tree binary. QEMU can generate one for you with the following command:
 
 
$ /local/path/qemu.git/aarch64-softmmu/qemu-system-aarch64 \
 
 
-machine virt,gic_version=3 \
 
-machine virt,gic_version=3 \
 
-machine virtualization=true \
 
-machine virtualization=true \
 
-cpu cortex-a57 -machine type=virt \
 
-cpu cortex-a57 -machine type=virt \
 
-smp 4 -m 4096 -display none \
 
-smp 4 -m 4096 -display none \
-machine dumpdtb=/mnt/virt-gicv3.dtb.dtb
+
-machine dumpdtb=virt-gicv3.dtb
  +
$ scp -P2222 virt-gicv3.dtb root@127.0.0.1:/boot/efi
 
We have written everything we need to the disk image. Let's umount /mnt and proceed to download the UEFI firmware binary from Linaro: [http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_CLANG35/QEMU_EFI.fd].
 
   
  +
We have written everything we need to the disk image. Let's poweroff the virtual machine and proceed to download the UEFI firmware binary from Linaro: [http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_CLANG35/QEMU_EFI.fd].
   
 
== Run QEMU ==
 
== Run QEMU ==
   
The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.
+
The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM, booting from EFI. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.
   
$ /path/to/qemu.git/aarch64-softmmu/qemu-system-aarch64 \
+
$ qemu-system-aarch64 \
 
-machine virt,gic_version=3 \
 
-machine virt,gic_version=3 \
 
-machine virtualization=true \
 
-machine virtualization=true \
Line 68: Line 85:
 
-bios /path/to/QEMU_EFI.bin \
 
-bios /path/to/QEMU_EFI.bin \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0
+
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0 -boot order=d
  +
  +
The boot order option will enable to get to UEFI prompt. Typing "FS0:" and then "xen" will boot hypervisor.
  +
  +
The Xen tools can be built further using below links :
  +
  +
https://wiki.debian.org/Arm64Qemu
  +
https://wiki.debian.org/QemuUserEmulation "Appendix: chrooting into target file systems"
  +
  +
= External References =
  +
  +
There is a nice [https://web.eecs.umich.edu/~jcma/blog/cjrl7dk7d000brlqjrv1rt7ru/ blog] entry covering the same topic.

Latest revision as of 12:46, 24 April 2019

QEMU AArch64 Emulator

QEMU is an Open Source GPLv2 software emulator. It can emulate a large range of machines of different architectures, including Cortex A57s based platforms.

The following steps help you setup QEMU to emulate an ARM64 machine and run Xen inside it.

Build QEMU

QEMU v4.0.0 can run Xen out of the box. Build QEMU like this:

$ ./configure --target-list=aarch64-softmmu --prefix=/usr/local
$ make
$ sudo make install

Install Xen

QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like xenial-server-cloudimg-arm64-uefi1.img.

Then, clone Linux and build the kernel using the defconfig for arm64 (make defconfig && make Image.gz). For building Linux kernel for ARM64 / ARM user might need cross compilers. Below are details with respect to different distros :

opensuse : https://opensuse.pkgs.org/tumbleweed/opensuse-oss/cross-aarch64-gcc7-7.3.1+r258313-1.2.x86_64.rpm.html

ubuntu : https://packages.ubuntu.com/en/trusty/gcc-aarch64-linux-gnu

Other distros : https://pkgs.org/

Steps can be found at http://events17.linuxfoundation.org/sites/events/files/slides/Shuah_Khan_cross_compile_linux.pdf

Run QEMU the first time, booting Linux directly (replace the MAC_ADDRESS and the paths). The following command uses user-networking and forwards port 2222 from the host to port 22 inside the virtual machine:

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt -nographic \
   -smp 4 -m 4000 \
   -kernel /path/to/linux.git/arch/arm64/boot/Image.gz --append "console=ttyAMA0 root=/dev/vda1 init=/bin/sh" \
   -netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
   -drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0

This command will give a shell prompt, where user might have to mount the root file system using below command :

 $ mount -o remount,rw /dev/vda1 /

Once mounted, user would like to set the 'root' password using 'passwd' command and then relaunch the virtual machine using above qemu command, but now remove 'init=/bin/sh'.

Once login with 'root' user, you may need to set the ssh authorized_keys of host machine, to access the virtual machine via remote.

Follow Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM to build Xen on ARM. Then, copy the Xen on ARM efi binary and the Linux kernel to the image:

 $ scp -P2222 /path/to/linux.git/arch/arm64/boot/Image.gz root@127.0.0.1:/boot/efi/kernel
 $ scp -P2222 /path/to/xen.git/xen/xen.efi root@127.0.0.1:/boot/efi

Let's write a config file for it:

 $ vi /boot/efi/xen.cfg

Let's use the following config:

 options=console=dtuart noreboot dom0_mem=512M
 kernel=kernel root=/dev/vda1 init=/bin/sh rw console=hvc0
 dtb=virt-gicv3.dtb

Now we need the device tree binary, "virt-gicv3.dtb". QEMU can generate one for you with the following command:

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 \
   -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt \
   -smp 4 -m 4096 -display none \
   -machine dumpdtb=virt-gicv3.dtb
 $ scp -P2222 virt-gicv3.dtb root@127.0.0.1:/boot/efi

We have written everything we need to the disk image. Let's poweroff the virtual machine and proceed to download the UEFI firmware binary from Linaro: [1].

Run QEMU

The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM, booting from EFI. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 \
   -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt \
   -smp 4 -m 4096 -display none \
   -serial mon:stdio \
   -bios /path/to/QEMU_EFI.bin \
   -netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
   -drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0 -boot order=d

The boot order option will enable to get to UEFI prompt. Typing "FS0:" and then "xen" will boot hypervisor.

The Xen tools can be built further using below links :

https://wiki.debian.org/Arm64Qemu https://wiki.debian.org/QemuUserEmulation "Appendix: chrooting into target file systems"

External References

There is a nice blog entry covering the same topic.