Difference between revisions of "Xen ARM with Virtualization Extensions/Allwinner"

From Xen
(arm: sunxi: add know issue. xen boot warning.)
(Hardware: newer Allwinner SoC support has been merged)
 
(20 intermediate revisions by 7 users not shown)
Line 1: Line 1:
  +
== Hardware ==
this wiki talks about xen on [http://linux-sunxi.org/A20 A20](Cortex-A7 dual core, code name sun7i). [http://linux-sunxi.org/A31 A31](Cortex-A7 quad core, code name sun6i) should work as well.
 
   
  +
This wiki talks about running Xen on systems with an Allwinner [http://linux-sunxi.org SoC] (often called sunxi) with virtualization capable cores (ARM Cortex-A7 or Cortex-A53).
== Compile u-boot ==
 
as described in [http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions#Firmware_Requirements Firmware_Requirements], xen need boot from non-secure hyper mode.
 
   
  +
This is well tested and supported on [http://linux-sunxi.org/A20 A20 SoCs] (Cortex-A7 dual core, sun7i). Since v4.10-rc1 there is support for more SoCs, including arm64 capable chips (A64, H5).
=== get the code ===
 
a worked branch is here:
 
https://github.com/bjzhang/u-boot-sunxi/tree/sunxi_hyp
 
   
  +
The following boards have been tested:
here is the details:
 
  +
; merge the hypmode branch
 
  +
* A20
git://git.linaro.org/people/aprzywara/u-boot.git hypmode_v4
 
  +
** [http://linux-sunxi.org/Cubieboard2#Cubieboard2_.28A20.29 Cubieboard2] (note that the original Cubieboard does not support virtualisation)
branch into
 
  +
*** [http://openmirage.org/wiki/xen-on-cubieboard2 Cubieboard2 with Xen: step-by-step guide]
git://github.com/linux-sunxi/u-boot-sunxi.git u-boot-sunxi
 
  +
*** [http://www.sourcediver.org/blog/2014/06/19/running-arch-linux-arm-with-xen-on-cubieboard-2/ Cubieboard 2 with Arch Linux ARM as Dom0 and DomU]
i tested hypmode_v4 with commit 3d2fc4e8 in u-boot-sunxi. hypmode_v5 should work as well.
 
  +
** [http://linux-sunxi.org/Cubietruck Cubietruck] (AKA Cubieboard 3)
; add [http://lists.xen.org/archives/html/xen-devel/2013-08/msg03040.html Ian Campbell patches] on it.
 
  +
* H2+/H3
; enable arm generic timer for sunxi, xen dom0 kernel need this:
 
  +
** [http://linux-sunxi.org/Orange_Pi_Zero OrangePi Zero]
diff --git a/include/configs/sun7i.h b/include/configs/sun7i.h
 
  +
* A64
index f0d9222..473d1eb 100644
 
  +
** [http://linux-sunxi.org/Pine64 Pine64]
--- a/include/configs/sun7i.h
 
  +
* H5
+++ b/include/configs/sun7i.h
 
  +
** [http://linux-sunxi.org/Orange_Pi_PC_2 OrangePi PC 2]
@@ -36,6 +36,8 @@
 
  +
 
  +
== Bootloader ==
#define CONFIG_ARMV7_VIRT
 
  +
 
  +
As described in [[Xen_ARM_with_Virtualization_Extensions#Firmware_Requirements|Firmware Requirements]], Xen needs to be booted in non-secure HYP mode (EL2 on ARM64).
+#define CONFIG_SYS_CLK_FREQ 24000000
 
  +
+
 
  +
Since at least v2014.11 U-Boot supports all the required technologies to support Xen: it supports most boards using an Allwinner SoC, boots the kernel (Xen in our case) in HYP mode and provides an PSCI interface for SMP bringup. There are no special options needed to get a Xen-capable U-Boot, just configure it for your board and build normally. Normal firmware builds (for instance as provided by distributions) should just work.
#if defined(CONFIG_SYS_SECONDARY_ON)
 
  +
#define CONFIG_BOARD_POSTCLK_INIT 1
 
  +
== Linux ==
#endif
 
   
  +
=== Upstream Support ===
refernce: [http://lists.xen.org/archives/html/xen-devel/2013-08/msg03040.html bootwrapper can't be compiled for cubieboard2]
 
   
  +
Support for the Allwinner processors in Linux is mainly a community effort, see [http://linux-sunxi.org/Main_Page the Linux Sunxi Wiki]. Support for Allwinner SoCs in the mainline kernel is constantly improving, most SoCs are well supported for a headless setup (serial, MMC, SATA, USB, Ethernet). Check the [http://linux-sunxi.org/Mainlining_Effort latest status].
=== Compile ===
 
for sd card
 
make Cubieboard2 CROSS_COMPILE=arm-linux-gnueabihf- -j8
 
for usb boot
 
make Cubieboard2_FEL CROSS_COMPILE=arm-linux-gnueabihf- -j8
 
   
  +
{{WarningLeft|The sunxi project used to maintain a Linux 3.4 fork based upon AllWinner's SDK. This and the original Allwinner provided BSP kernel port are not usable with Xen.}}
== Building Linux Dom0 ==
 
   
  +
=== Building ===
First, we will build Linux for dom0 to have a device tree to provide to Xen.
 
   
  +
None of the kernel provided defconfigs for ARM(32) include the required Xen options. To get a Xen capable kernel, use either multi_v7_defconfig or sunxi_defconfig as a base or use your distribution provided config file.
You can get a tree from [https://github.com/bjzhang/linux-allwinner/tree/sun7i-xen-dom0 sun7i-xen-dom0] which is based on arokux sunxi usb branch.
 
  +
From here you will want to enable the various Xen options (CONFIG_XEN, frontend and backend drivers etc, see [[Mainline_Linux_Kernel_Configs]] for more general details).
the emac patch in this tree is needed for enable network in dom0.
 
i am confuse about what does "CONFIG XEN" mean. it will check the kernel features for dom0 and domU kernel.
 
if you want to a dom0 kernel, you need to uncheck all the FRONTEND config, check CONFIG_XEN_BACKEND, and check the BACKEND config as you need:
 
CONFIG_XEN_BLKDEV_BACKEND=y
 
CONFIG_XEN_NETDEV_BACKEND=y
 
   
  +
Then:
or use sunxi dom0 kernel directly
 
git clone -b sun7i-xen-dom0 https://github.com/bjzhang/linux-allwinner.git sun7i-xen-dom0
 
cd sun7i-xen-dom0
 
make sun7i_dom0_defconfig
 
 
make zImage
 
make zImage
 
make dtbs
 
make dtbs
   
The device tree is located in arch/arm/boot/dts/sun7i-a20-cubieboard2-xen.dtb
+
The final kernel binary will be in `arch/arm/boot/zImage` and the device tree will be located in `arch/arm/boot/dts` e.g. sun7i-a20-cubieboard2.dtb, sun7i-a20-cubietruck.dtb etc.
  +
  +
The arm64 defconfig includes the Xen options since Linux v4.5, so most kernels should work out of the box.
  +
  +
The same kernel image should work for both dom0 and domU as well as for bare-metal boots.
   
 
== Building Xen ==
 
== Building Xen ==
   
the lastest xen should work. reference [[Xen_ARMv7_with_Virtualization_Extensions#Building_Xen_on_ARM|Build Xen on ARM]] and [[Xen_ARMv7_with_Virtualization_Extensions#Building Xen and Linux Dom0|Building Xen and Linux Dom0]] for building xen and generating the xen-uImage.
+
The latest xen should work. See [[Xen_ARMv7_with_Virtualization_Extensions#Building_Xen_on_ARM|Build Xen on ARM]] and [[Xen_ARMv7_with_Virtualization_Extensions#Building Xen and Linux Dom0|Building Xen and Linux Dom0]] for building xen and generating the xen-uImage.
i test pass on xen commit 720f45ad: Julien Grall, xen/evtchn: Fix build on ARM
 
   
use the following build command for enable early printk for a20:
+
Use the following build command for enable early printk for a20:
make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=sun7i CONFIG_DTB_FILE=/path/to/sun7i-a20-cubieboard2-xen.dtb
+
make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=sun7i
   
== Building linux DomU ==
+
== Booting Xen and Dom0 ==
  +
=== Command lines ===
   
  +
For Xen the correct command line for console on the first serial port is `dtuart=/soc@01c00000/serial@01c28000`. For other serial ports use the correct name/address from the device tree.
xen domU kernel is supported by upstream. and remember that domU guest do not aware of which hardware it running on, it only know about the mach-virt.
 
if you want to a domU kernel, you need to ucheck all the BACKEND config and check the FRONTEND config as you need:
 
CONFIG_XEN_BLKDEV_FRONTEND=y
 
CONFIG_XEN_NETDEV_FRONTEND=y
 
CONFIG_HVC_XEN_FRONTEND=y
 
CONFIG_XEN_FBDEV_FRONTEND=y
 
note that '''CONFIG_XEN_XENBUS_FRONTEND''' will be automatically checked by any FRONTEND driver.
 
   
  +
For dom0 an additional `clk_ignore_unused` option must be included in the dom0 kernel command line.
currenly, xen directly use the dom0 gic dist base and cpu base as domU gic dist and cpu base. reference gicv_setup() in xen/arch/arm/gic.c for details.
 
for sunxi, need chances the followings: (this patch is included in my branch)
 
   
  +
=== Load addresses ===
diff --git a/arch/arm/boot/dts/xenvm-4.2.dts b/arch/arm/boot/dts/xenvm-4.2.dts
 
index 3369151..5d5689e 100644
 
--- a/arch/arm/boot/dts/xenvm-4.2.dts
 
+++ b/arch/arm/boot/dts/xenvm-4.2.dts
 
@@ -50,13 +50,13 @@
 
reg = <0 0x80000000 0 0x08000000>;
 
};
 
 
- gic: interrupt-controller@2c001000 {
 
+ gic: interrupt-controller@1c81000{
 
compatible = "arm,cortex-a15-gic", "arm,cortex-a9-gic";
 
#interrupt-cells = <3>;
 
#address-cells = <0>;
 
interrupt-controller;
 
- reg = <0 0x2c001000 0 0x1000>,
 
- <0 0x2c002000 0 0x100>;
 
+ reg = <0 0x01c81000 0 0x1000>,
 
+ <0 0x01c82000 0 0x1000>;
 
};
 
 
timer {
 
   
  +
Due to [http://wiki.xen.org/wiki/Xen_ARM_TODO#Domain_0_memory_limitation_due_to_1:1_mapping Xen ARM TODO/Domain 0 memory limitation due to 1:1 mapping] it is recommended to place the initial binary blobs towards the top of RAM, leaving 2MB free for Xen to relocate into.
it should not need to change after ian auto-generate dts patch upstreamed.
 
   
  +
The following table shows suggested load addresses for 1GB (e.g cubieboard2) and 2GB (e.g. cubietruck) systems. These can be used with the <tt>boot.scr</tt> boot script described in the next section.
git clone -b sun7i_xen_domU https://github.com/bjzhang/linux-allwinner.git sun7i_xen_domU
 
cd sun7i_xen_domU
 
make sun7i_domU_defconfig
 
make zImage
 
make dtbs
 
   
  +
{|
== Booting Xen and Dom0 ==
 
  +
!Binary
  +
!1GB systems
  +
!2GB systems
  +
!Size
  +
!<tt>boot.scr</tt> variable name
  +
|-
  +
|Top of RAM
  +
|<tt>0x80000000</tt>
  +
|<tt>0xc0000000</tt>
  +
|
  +
|
  +
|-
  +
|Xen relocation target address
  +
|<tt>0x7fe00000</tt>
  +
|<tt>0xbfe00000</tt>
  +
|2MB
  +
|
  +
|-
  +
|Dom0 kernel (Linux vmlinuz)
  +
|<tt>0x7f600000</tt>
  +
|<tt>0xbf600000</tt>
  +
|8MB
  +
|<tt>kernel_addr_r</tt>
  +
|-
  +
|Dom0 initial ramdisk (optional)
  +
|<tt>0x7ee00000</tt>
  +
|<tt>0xbee00000</tt>
  +
|8MB
  +
|<tt>ramdisk_addr_r</tt>
  +
|-
  +
|Device tree blob
  +
|<tt>0x7ec00000</tt>
  +
|<tt>0xbec00000</tt>
  +
|2MB
  +
|<tt>fdt_addr</tt>
  +
|-
  +
|Xen initial load address
  +
|<tt>0x7ea00000</tt>
  +
|<tt>0xbea00000</tt>
  +
|2MB
  +
|<tt>xen_addr_r</tt>
  +
|}
   
  +
This assumes that the kernel is <8MB, if it is larger then shift everything down in 2MB increments. Likewise for the initial ramdisk.
To boot Xen and Dom0, you can use USB boot or sd boot.
 
  +
  +
=== Boot script ===
  +
  +
To boot Xen and Dom0, you can use USB boot or sd boot. Some runtime manipulation of the DTB is needed, but the basic outline is the same in each case (only the method to load the binary differs).
  +
  +
# SUNXI Xen Boot Script
  +
  +
# Addresses suitable for 1GB system, adjust as appropriate for a 2GB system.
  +
# Top of RAM: 0x80000000
  +
# Xen relocate addr 0x7fe00000
  +
setenv kernel_addr_r 0x7f600000 # 8M
  +
setenv ramdisk_addr_r 0x7ee00000 # 8M
  +
setenv fdt_addr 0x7ec00000 # 2M
  +
setenv xen_addr_r 0x7ea00000 # 2M
  +
  +
setenv fdt_high 0xffffffff # Load fdt in place instead of relocating
  +
  +
# Load xen/xen to ${xen_addr_r}. e.g. tftp, fatload or ext2load to ${xen_addr_r}.
  +
# see the following sections for details of booting from various devices.
  +
  +
setenv bootargs "console=dtuart dtuart=/soc@01c00000/serial@01c28000 dom0_mem=128M"
  +
  +
# Load appropriate .dtb file to ${fdt_addr} e.g. tftp, fatload or ext2load to ${fdt_addr}.
  +
# see the following sections for details of booting from various devices.
  +
  +
fdt addr ${fdt_addr} 0x40000
  +
  +
fdt resize
  +
  +
fdt chosen
  +
  +
fdt set /chosen \#address-cells <1>
  +
fdt set /chosen \#size-cells <1>
  +
  +
# Load Linux arch/arm/boot/zImage to ${kernel_addr_r}. e.g. tftp, fatload or ext2load to ${kernel_addr_r}.
  +
# see the following sections for details of booting from various devices.
  +
  +
fdt mknod /chosen module@0
  +
fdt set /chosen/module@0 compatible "xen,linux-zimage" "xen,multiboot-module"
  +
fdt set /chosen/module@0 reg <${kernel_addr_r} 0x${filesize} >
  +
fdt set /chosen/module@0 bootargs "console=hvc0 ro root=/dev/sda1 clk_ignore_unused"
  +
  +
bootz ${xen_addr_r} - ${fdt_addr}
  +
  +
The easiest approach is to create a file (boot.xen) with this and then convert this into a u-boot boot.scr with:
  +
  +
mkimage -A arm -T script -d boot.xen boot.scr.
  +
  +
This can then be loaded to address 0x41000000 and then launched with:
  +
  +
source 0x41000000
   
 
=== Booting via PXE ===
 
=== Booting via PXE ===
  +
TODO
 
  +
To boot use the boot.scr above loading images with:
  +
  +
tftp ${address} /path/to/image
  +
  +
For example:
  +
  +
# Load xen/xen to ${xen_addr_r}
  +
tftp ${xen_addr_r} /cubieboard/xen
  +
  +
# Load appropriate .dtb file to ${fdt_addr}
  +
tftp ${fdt_addr} /cubieboard/sun7i-a20-cubieboard2.dtb
  +
  +
# Load Linux arch/arm/boot/zImage to ${kernel_addr_r}
  +
tftp ${kernel_addr_r} /cubieboard/vmlinuz
  +
  +
To boot this set bootcmd:
  +
  +
setenv bootcmd setenv autoload no\;dhcp\;tftp 0x41000000 /boot.scr;source 0x41000000
  +
saveenv
  +
boot
   
 
=== Booting via USBBoot(FEL) ===
 
=== Booting via USBBoot(FEL) ===
  +
reference: http://linux-sunxi.org/FEL/USBBoot
 
  +
reference: http://linux-sunxi.org/FEL/USBBoot
  +
  +
Arrange for all of the images to be loaded at the appropriate addresses via the FEL utility and then load a suitable boot.scr to launch them.
   
 
=== Booting directly on the SD card ===
 
=== Booting directly on the SD card ===
reference [http://linux-sunxi.org/Bootable_SD_card] for how to make a bootable sdcard.
 
   
  +
reference [http://linux-sunxi.org/Bootable_SD_card Bootable SD Card] for how to make a bootable sdcard.
* Copy xen-uImage, zImage and the initrd the root directory of you SD card.
 
  +
* save the bootcmd
 
  +
Copy xen, zImage, the dtb and boot.scr to the root directory of you SD card.
setenv bootcmd "fatload mmc 0 45000000 xen-uImage; fatload mmc 0 50000000 zImage; fatload mmc 0 60000000 initrd; bootm 45000000"
 
  +
savenv
 
  +
To boot use the boot.scr above loading images with:
* boot
 
  +
boot
 
  +
fatload mmc 0 ${address} /path/to/image
  +
  +
For example:
  +
  +
# Load xen/xen to ${xen_addr_r}
  +
fatload mmc 0 ${xen_addr_r} /xen
  +
  +
# Load appropriate .dtb file to ${fdt_addr}
  +
fatload mmc 0 ${fdt_addr} /sun7i-a20-cubieboard2.dtb
  +
  +
# Load Linux arch/arm/boot/zImage to ${kernel_addr_r}
  +
fatload mmc 0 ${kernel_addr_r} /vmlinuz
  +
  +
To boot set the bootcmd:
  +
  +
setenv bootcmd "fatload mmc 0 0x41000000 boot.scr; source 0x41000000"
  +
savenv
  +
boot
  +
  +
This assume the SD card has only 1 partition with '''fat''' filesystem. Use e.g. ext2load etc if this is not the case.
  +
  +
=== Mount NFS after initrd boot ===
   
  +
Using a [http://download.opensuse.org/repositories/devel:/ARM:/12.3:/Contrib:/sunxi/images/openSUSE-12.3-ARM-JeOS-rootfs.armv7l-1.12.1-Build71.1.tbz distrbution] is easily to compile xen tools.
Assume the SD card has only 1 partition with '''fat''' filesystem.
 
   
  +
Both opensuse arm 12.3 rootfs and Debian have been used by developers. Other distribution which support armhf is ok too.
== Mount NFS after initrd boot ==
 
Using a [http://download.opensuse.org/repositories/devel:/ARM:/12.3:/Contrib:/sunxi/images/openSUSE-12.3-ARM-JeOS-rootfs.armv7l-1.12.1-Build71.1.tbz distrbution] is easily to compile xen tools. i use opensuse arm 12.3 rootfs for it. other distribution which support armhf is ok too.
 
   
 
== Create domU ==
 
== Create domU ==
Line 141: Line 236:
 
;domU config example
 
;domU config example
 
> cat domU_test
 
> cat domU_test
kernel = "/root/domU/zImage_dtb"
+
kernel = "/root/domU/zImage
 
memory = "128"
 
memory = "128"
 
name = "domU"
 
name = "domU"
Line 158: Line 253:
   
 
== DomU kernel debug ==
 
== DomU kernel debug ==
  +
xen support varity debug method. you could use xl debug-keys for dumping information. use xen_raw_console_write for output DomU kernel output.
 
  +
Xen supports a variety of debug methods. You can use xl debug-keys for dumping information. Use `xen_raw_console_write` for output DomU kernel output.
but gdbsx is not work for arm right now.
 
  +
  +
`gdbsx` is not work for arm right now.
   
 
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
 
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
Line 185: Line 282:
 
[http://linux-sunxi.org/ allwinner sunxi resources]
 
[http://linux-sunxi.org/ allwinner sunxi resources]
 
[http://lists.xen.org/archives/html/xen-devel/2013-08/msg03040.html]
 
[http://lists.xen.org/archives/html/xen-devel/2013-08/msg03040.html]
 
== Known Issue ==
 
without [http://lists.xen.org/archives/html/xen-devel/2013-09/msg02232.html sunxi platform patch], there is a warning during xen boot:
 
(XEN) WARNING: Unrecognized/unsupported device tree compatible list
 
(XEN) Available platform support:
 
(XEN) - VERSATILE EXPRESS
 
(XEN) - SAMSUNG EXYNOS5
 
(XEN) - CALXEDA MIDWAY
 
(XEN) - TI OMAP5
 
   
 
[[Category:XenARM]]
 
[[Category:XenARM]]
 
[[Category:Developers]]
 
[[Category:Developers]]
 
[[Category:Xen 4.4]]
 
[[Category:Xen 4.4]]
  +
[[Category:Xen 4.5]]

Latest revision as of 19:29, 17 October 2017

Hardware

This wiki talks about running Xen on systems with an Allwinner SoC (often called sunxi) with virtualization capable cores (ARM Cortex-A7 or Cortex-A53).

This is well tested and supported on A20 SoCs (Cortex-A7 dual core, sun7i). Since v4.10-rc1 there is support for more SoCs, including arm64 capable chips (A64, H5).

The following boards have been tested:

Bootloader

As described in Firmware Requirements, Xen needs to be booted in non-secure HYP mode (EL2 on ARM64).

Since at least v2014.11 U-Boot supports all the required technologies to support Xen: it supports most boards using an Allwinner SoC, boots the kernel (Xen in our case) in HYP mode and provides an PSCI interface for SMP bringup. There are no special options needed to get a Xen-capable U-Boot, just configure it for your board and build normally. Normal firmware builds (for instance as provided by distributions) should just work.

Linux

Upstream Support

Support for the Allwinner processors in Linux is mainly a community effort, see the Linux Sunxi Wiki. Support for Allwinner SoCs in the mainline kernel is constantly improving, most SoCs are well supported for a headless setup (serial, MMC, SATA, USB, Ethernet). Check the latest status.

Icon Ambox.png The sunxi project used to maintain a Linux 3.4 fork based upon AllWinner's SDK. This and the original Allwinner provided BSP kernel port are not usable with Xen.


Building

None of the kernel provided defconfigs for ARM(32) include the required Xen options. To get a Xen capable kernel, use either multi_v7_defconfig or sunxi_defconfig as a base or use your distribution provided config file. From here you will want to enable the various Xen options (CONFIG_XEN, frontend and backend drivers etc, see Mainline_Linux_Kernel_Configs for more general details).

Then:

 make zImage
 make dtbs

The final kernel binary will be in `arch/arm/boot/zImage` and the device tree will be located in `arch/arm/boot/dts` e.g. sun7i-a20-cubieboard2.dtb, sun7i-a20-cubietruck.dtb etc.

The arm64 defconfig includes the Xen options since Linux v4.5, so most kernels should work out of the box.

The same kernel image should work for both dom0 and domU as well as for bare-metal boots.

Building Xen

The latest xen should work. See Build Xen on ARM and Building Xen and Linux Dom0 for building xen and generating the xen-uImage.

Use the following build command for enable early printk for a20:

   make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=sun7i

Booting Xen and Dom0

Command lines

For Xen the correct command line for console on the first serial port is `dtuart=/soc@01c00000/serial@01c28000`. For other serial ports use the correct name/address from the device tree.

For dom0 an additional `clk_ignore_unused` option must be included in the dom0 kernel command line.

Load addresses

Due to Xen ARM TODO/Domain 0 memory limitation due to 1:1 mapping it is recommended to place the initial binary blobs towards the top of RAM, leaving 2MB free for Xen to relocate into.

The following table shows suggested load addresses for 1GB (e.g cubieboard2) and 2GB (e.g. cubietruck) systems. These can be used with the boot.scr boot script described in the next section.

Binary 1GB systems 2GB systems Size boot.scr variable name
Top of RAM 0x80000000 0xc0000000
Xen relocation target address 0x7fe00000 0xbfe00000 2MB
Dom0 kernel (Linux vmlinuz) 0x7f600000 0xbf600000 8MB kernel_addr_r
Dom0 initial ramdisk (optional) 0x7ee00000 0xbee00000 8MB ramdisk_addr_r
Device tree blob 0x7ec00000 0xbec00000 2MB fdt_addr
Xen initial load address 0x7ea00000 0xbea00000 2MB xen_addr_r

This assumes that the kernel is <8MB, if it is larger then shift everything down in 2MB increments. Likewise for the initial ramdisk.

Boot script

To boot Xen and Dom0, you can use USB boot or sd boot. Some runtime manipulation of the DTB is needed, but the basic outline is the same in each case (only the method to load the binary differs).

   # SUNXI Xen Boot Script
   
   # Addresses suitable for 1GB system, adjust as appropriate for a 2GB system.
   # Top of RAM:         0x80000000
   # Xen relocate addr   0x7fe00000
   setenv kernel_addr_r  0x7f600000 # 8M
   setenv ramdisk_addr_r 0x7ee00000 # 8M
   setenv fdt_addr       0x7ec00000 # 2M
   setenv xen_addr_r     0x7ea00000 # 2M
   
   setenv fdt_high      0xffffffff # Load fdt in place instead of relocating
   
   # Load xen/xen to ${xen_addr_r}. e.g. tftp, fatload or ext2load to ${xen_addr_r}.
   # see the following sections for details of booting from various devices.
   
   setenv bootargs "console=dtuart dtuart=/soc@01c00000/serial@01c28000 dom0_mem=128M"
   
   # Load appropriate .dtb file to ${fdt_addr} e.g. tftp, fatload or ext2load to ${fdt_addr}.
   # see the following sections for details of booting from various devices.
   
   fdt addr ${fdt_addr} 0x40000
   
   fdt resize
   
   fdt chosen
   
   fdt set /chosen \#address-cells <1>
   fdt set /chosen \#size-cells <1>
   
   # Load Linux arch/arm/boot/zImage to ${kernel_addr_r}. e.g. tftp, fatload or ext2load to ${kernel_addr_r}.
   # see the following sections for details of booting from various devices.
   
   fdt mknod /chosen module@0
   fdt set /chosen/module@0 compatible "xen,linux-zimage" "xen,multiboot-module"
   fdt set /chosen/module@0 reg <${kernel_addr_r} 0x${filesize} >
   fdt set /chosen/module@0 bootargs "console=hvc0 ro root=/dev/sda1 clk_ignore_unused"
   
   bootz ${xen_addr_r} - ${fdt_addr}

The easiest approach is to create a file (boot.xen) with this and then convert this into a u-boot boot.scr with:

   mkimage -A arm -T script -d boot.xen boot.scr.

This can then be loaded to address 0x41000000 and then launched with:

   source 0x41000000

Booting via PXE

To boot use the boot.scr above loading images with:

   tftp ${address} /path/to/image

For example:

    # Load xen/xen to ${xen_addr_r}
    tftp ${xen_addr_r} /cubieboard/xen
   
    # Load appropriate .dtb file to ${fdt_addr}
    tftp ${fdt_addr} /cubieboard/sun7i-a20-cubieboard2.dtb
   
    # Load Linux arch/arm/boot/zImage to ${kernel_addr_r}
    tftp ${kernel_addr_r} /cubieboard/vmlinuz

To boot this set bootcmd:

   setenv bootcmd setenv autoload no\;dhcp\;tftp 0x41000000 /boot.scr;source 0x41000000
   saveenv
   boot

Booting via USBBoot(FEL)

reference: http://linux-sunxi.org/FEL/USBBoot

Arrange for all of the images to be loaded at the appropriate addresses via the FEL utility and then load a suitable boot.scr to launch them.

Booting directly on the SD card

reference Bootable SD Card for how to make a bootable sdcard.

Copy xen, zImage, the dtb and boot.scr to the root directory of you SD card.

To boot use the boot.scr above loading images with:

   fatload mmc 0 ${address} /path/to/image

For example:

   # Load xen/xen to ${xen_addr_r}
   fatload mmc 0 ${xen_addr_r} /xen
   
   # Load appropriate .dtb file to ${fdt_addr}
   fatload mmc 0 ${fdt_addr} /sun7i-a20-cubieboard2.dtb
   
   # Load Linux arch/arm/boot/zImage to ${kernel_addr_r}
   fatload mmc 0 ${kernel_addr_r} /vmlinuz

To boot set the bootcmd:

   setenv bootcmd "fatload mmc 0 0x41000000 boot.scr; source 0x41000000"
   savenv
   boot

This assume the SD card has only 1 partition with fat filesystem. Use e.g. ext2load etc if this is not the case.

Mount NFS after initrd boot

Using a distrbution is easily to compile xen tools.

Both opensuse arm 12.3 rootfs and Debian have been used by developers. Other distribution which support armhf is ok too.

Create domU

setup the harddisk
   losetup /dev/loop0 /root/domU/rootfs.ext3 
domU config example
   > cat domU_test
   kernel = "/root/domU/zImage
   memory = "128" 
   name = "domU" 
   vcpus = 1 
   serial="pty"
   disk = [ 'phy:/dev/loop0,xvda,w' ]
   vif=[ 'mac=00:16:3e:56:af:69,bridge=virbr0,type=netfront', ]
create
   xl create domU_test

or enable all the debug message

   xl -vvv create -d domU_testa
connect to console

xl console domU_test

DomU kernel debug

Xen supports a variety of debug methods. You can use xl debug-keys for dumping information. Use `xen_raw_console_write` for output DomU kernel output.

`gdbsx` is not work for arm right now.

   diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
   index b4e8500..060c163 100644
   --- a/kernel/printk/printk.c
   +++ b/kernel/printk/printk.c
   @@ -1488,6 +1488,7 @@ static size_t cont_print_text(char *text, size_t size)
       return textlen;
    }
    
   +#include <xen/hvc-console.h>
    asmlinkage int vprintk_emit(int facility, int level,
                   const char *dict, size_t dictlen,
                   const char *fmt, va_list args)
   @@ -1546,6 +1547,7 @@ asmlinkage int vprintk_emit(int facility, int level,
        * prefix which might be passed-in as a parameter.
        */
       text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
   +    xen_raw_console_write(text);
    
       /* mark and strip a trailing newline */
       if (text_len && text[text_len-1] == '\n') {

Resources

allwinner sunxi resources [1]