Difference between revisions of "Tuning Xen for Performance"

From Xen
(HAP vs. shadow)
(Tuning your Xen installation: recommended settings)
Line 5: Line 5:
  
 
=== Memory ===
 
=== Memory ===
If the dom0 Host-to-be has more memory than a typical laptop/desktop system has, then do not rely on dom0 ballooning.
+
If the host has more memory than a typical laptop/desktop system, then do not rely on dom0 ballooning.
Instead set the dom0 ram to something between 1 and 4GB adding dom0_mem=2048M to the Xen command line.
+
Instead set the dom0 memory to be something between 1 and 4GB adding dom0_mem=1024M to the Xen command line.
  
 
1GB is enough for a pretty large host, more will be needed if you expect your users to use advanced storage types as ZFS or distributed filesystems.
 
1GB is enough for a pretty large host, more will be needed if you expect your users to use advanced storage types as ZFS or distributed filesystems.
Line 14: Line 14:
  
 
=== Dom0 VCPUs ===
 
=== Dom0 VCPUs ===
You might find that if you are running Xen on a large machine with more than 12 core, limiting the number of Dom0 vcpus to 8 and pinning them can improve performances. See: [[XenCommonProblems#Can_I_dedicate_a_cpu_core_.28or_cores.29_only_for_dom0.3F| Can I dedicate a cpu core (or cores) only for dom0? ]].
+
You might find that if you are running Xen on a large machine with more than 8 or 12 cores, limiting the number of Dom0 vcpus to 8 and pinning them can improve performances. See: [[XenCommonProblems#Can_I_dedicate_a_cpu_core_.28or_cores.29_only_for_dom0.3F| Can I dedicate a cpu core (or cores) only for dom0? ]].
  
  
 
=== HAP vs. shadow ===
 
=== HAP vs. shadow ===
HAP stands for hardware assisted paging and requires a CPU feature called EPT by Intel and RVI by AMD. It is used to manage the guest's mmu and it is usually faster than its fully software alternative (called shadow pagetabled). However some workloads characterized by quasi-static page tables perform better on shadow than HAP, for example specjbb.
+
HAP stands for hardware assisted paging and requires a CPU feature called EPT by Intel and RVI by AMD. It is used to manage the guest's MMU. The alternative is shadow paging, completely managed by Xen in software.
HAP can be enabled/disabled passing hap=0/1 in the VM config file.
+
On HAP TLB misses are expensive so if you have really random access, HAP will be expensive. For shadow, page table updates are expensive.
 +
HAP is enabled by default (and it is the recommended setting) but can be disabled passing hap=0 in the VM config file.
 +
 
  
 
=== PV vs PV on HVM ===
 
=== PV vs PV on HVM ===
Linux, NetBSD, FreeBSD and Solaris can run as PV or PV on HVM guests (see [[Xen_Beginners_Guide#Creating_a_Debian_PV_.28Paravirtualized.29_Guest | this page]] and [[Xen_Linux_PV_on_HVM_drivers#Example_HVM_guest_configuration_file_for_PVHVM_use | this page]] for more info).
+
Linux, NetBSD, FreeBSD and Solaris can run as PV or PV on HVM guests (see [[Xen_Beginners_Guide#Creating_a_Debian_PV_.28Paravirtualized.29_Guest | this page]] for instructions on how to install a Debian PV guest and [[Xen_Linux_PV_on_HVM_drivers#Example_HVM_guest_configuration_file_for_PVHVM_use | this page]] for an example PV on HVM VM config file).
 
Memory intensive workloads that involve the continuous creation and destruction of page tables can perform better when run in a PV on HVM guest. Examples are kernbench and sql-bench.
 
Memory intensive workloads that involve the continuous creation and destruction of page tables can perform better when run in a PV on HVM guest. Examples are kernbench and sql-bench.
 
On the other hand memory workloads that run on a quasi-static set of page tables run better on a PV guests. An example of this kind of workloads is specjbb.
 
On the other hand memory workloads that run on a quasi-static set of page tables run better on a PV guests. An example of this kind of workloads is specjbb.
 
See [[ Xen_Linux_PV_on_HVM_drivers#Performance_Tradeoffs ]] for more details.
 
See [[ Xen_Linux_PV_on_HVM_drivers#Performance_Tradeoffs ]] for more details.

Revision as of 15:08, 25 June 2012

Tuning your Xen installation: recommended settings

Storage options

LVM is the fastest storage backend you can choose. See Storage_options for more details.

Memory

If the host has more memory than a typical laptop/desktop system, then do not rely on dom0 ballooning. Instead set the dom0 memory to be something between 1 and 4GB adding dom0_mem=1024M to the Xen command line.

1GB is enough for a pretty large host, more will be needed if you expect your users to use advanced storage types as ZFS or distributed filesystems.

The reason to not give all ram to dom0 is that it takes time to re-purpose it went domUs are started. On a host with 128GB Ram you're getting things stuck for minutes while dom0 is ballooning down to make space.


Dom0 VCPUs

You might find that if you are running Xen on a large machine with more than 8 or 12 cores, limiting the number of Dom0 vcpus to 8 and pinning them can improve performances. See: Can I dedicate a cpu core (or cores) only for dom0? .


HAP vs. shadow

HAP stands for hardware assisted paging and requires a CPU feature called EPT by Intel and RVI by AMD. It is used to manage the guest's MMU. The alternative is shadow paging, completely managed by Xen in software. On HAP TLB misses are expensive so if you have really random access, HAP will be expensive. For shadow, page table updates are expensive. HAP is enabled by default (and it is the recommended setting) but can be disabled passing hap=0 in the VM config file.


PV vs PV on HVM

Linux, NetBSD, FreeBSD and Solaris can run as PV or PV on HVM guests (see this page for instructions on how to install a Debian PV guest and this page for an example PV on HVM VM config file). Memory intensive workloads that involve the continuous creation and destruction of page tables can perform better when run in a PV on HVM guest. Examples are kernbench and sql-bench. On the other hand memory workloads that run on a quasi-static set of page tables run better on a PV guests. An example of this kind of workloads is specjbb. See Xen_Linux_PV_on_HVM_drivers#Performance_Tradeoffs for more details.