Difference between revisions of "Archived/Xen Roadmap/4.3"

From Xen
(Added "Allow XSM to override IS_PRIV checks in the hypervisor")
Line 13: Line 13:
   
 
NB: Several of the items on this list marked (external). These are not part of the Xen tree, but are directly related to our users' experience (e.g., work in Linux or qemu) or to integration with other important projects (e.g., libvirt indings). Since all of these are part of the Xen community work, and comes from the same pool of labor, it makes sense to track the progress here, even though they won't explicitly be released as part of 4.3.
 
NB: Several of the items on this list marked (external). These are not part of the Xen tree, but are directly related to our users' experience (e.g., work in Linux or qemu) or to integration with other important projects (e.g., libvirt indings). Since all of these are part of the Xen community work, and comes from the same pool of labor, it makes sense to track the progress here, even though they won't explicitly be released as part of 4.3.
  +
  +
Meanings of prognoses:
  +
* Excellent: It would be very unlikely for this not to be finished in time.
  +
* Good: Everything is on track, and is likely to make it.
  +
* Fair: A pretty good chance of making it, but not as certain
  +
* Poor: Likely not to make it unless intervention is made
  +
* Not for 4.3: Self-explanatory
  +
   
 
=== Completed ===
 
=== Completed ===
Line 22: Line 30:
 
- pci pass-thru (external)
 
- pci pass-thru (external)
 
- enable dirtybit tracking during migration (external)
 
- enable dirtybit tracking during migration (external)
  +
- xl cd-{insert,eject} (external)
   
 
* CPUID-based idle (don't rely on ACPI info f/ dom0)
 
* CPUID-based idle (don't rely on ACPI info f/ dom0)
  +
  +
* Persistent grants for blk (external)
  +
- Linux
 
</pre>
 
</pre>
   
Line 46: Line 58:
 
owner: mukesh@oracle
 
owner: mukesh@oracle
 
status (Linux): 3rd draft patches posted.
 
status (Linux): 3rd draft patches posted.
status (Xen): Patches being cleaned up for submission
+
status (Xen): RFC submitted
  +
prognosis: Good
   
 
* Event channel scalability
 
* Event channel scalability
owner: attilio@citrix
+
owner: wei@citrix
status: initial design proposed
+
status: RFC submitted
  +
prognosis: Good
 
Increase limit on event channels (currently 1024 for 32-bit guests,
 
Increase limit on event channels (currently 1024 for 32-bit guests,
 
4096 for 64-bit guests)
 
4096 for 64-bit guests)
   
* ARM server port
+
* ARM v7 server port
 
owner: ijc@citrix
 
owner: ijc@citrix
  +
prognosis: Excellent
status: Core hypervisor patches accepted; Linux paches pending
 
  +
status: Core hypervisor and Linux patches accepted. Tools patches submitted.
  +
  +
* ARM v8 server port (tech preview)
  +
owner: ijc@citrix
  +
status: ?
  +
prognosis: Tech preview only
   
 
* NUMA scheduler affinity
 
* NUMA scheduler affinity
Line 62: Line 82:
 
owner: dario@citrix
 
owner: dario@citrix
 
status: Patches posted
 
status: Patches posted
  +
prognosis: Excellent
 
* Allow XSM to override IS_PRIV checks in the hypervisor
 
owner: Daniel De Graaf
 
status: patches against 4.3-unstable posted, awaiting approval
 
This makes it possible to allow some user domains limited access to
 
dom0-only hypercalls. This could be used to allow a user-created
 
toolstack domain to administer its own set of VMs instead of relying
 
on dom0's toolstack.
 
   
 
* NUMA Memory migration
 
* NUMA Memory migration
 
owner: dario@citrix
 
owner: dario@citrix
status: ?
+
status: in progress
  +
prognosis: Fair
   
* blktap3
+
* blktap3
 
owner: thanos@citrix
 
owner: thanos@citrix
status: in progress
+
status: RFCs posted
  +
prognosis: Not for 4.3
   
 
* Default to QEMU upstream
 
* Default to QEMU upstream
Line 84: Line 99:
 
owner: anthony@citrix
 
owner: anthony@citrix
 
status: in progress
 
status: in progress
  +
prognosis: ?
 
qemu-upstream needs a more fully-featured libc than exists in
 
qemu-upstream needs a more fully-featured libc than exists in
 
mini-os. Either work on a minimalist linux-based stubdom with
 
mini-os. Either work on a minimalist linux-based stubdom with
 
glibc, or port one of the BSD libcs to minios.
 
glibc, or port one of the BSD libcs to minios.
   
- xl cd-{insert,eject} (external)
+
* Persistent grants for blk (external)
  +
owner: roger@citrix
status: intilal implementation submitted
 
  +
status: qemu patches submitted
  +
prognosis: Good
   
* Persistent grants (external)
+
* Persistent grants for net
owner: roger.pau@citrix
+
owner: annie.li@citrix
 
status: Initial implementation posted
 
status: Initial implementation posted
  +
prognosis: ?
   
 
* Multi-page blk rings (external)
 
* Multi-page blk rings (external)
- blkback in kernel (konrad@oracle, ?@intel)
+
- blkback in kernel roger@citrix
 
- qemu blkback
 
- qemu blkback
status: Not started.
+
status: Starting soon
  +
prognosis: Fair
   
 
* Multi-page net protocol (external)
 
* Multi-page net protocol (external)
Line 108: Line 128:
 
* Scalability: 16TiB of RAM
 
* Scalability: 16TiB of RAM
 
owner: jan@suse
 
owner: jan@suse
status: Not started
+
status: In progress
  +
prognosis: Good
   
 
* vTPM updates
 
* vTPM updates
 
owner: Matthew Fioravante @ Johns Hopkins
 
owner: Matthew Fioravante @ Johns Hopkins
  +
prognosis: Good
status: v2 patches submitted
 
  +
status: some patches submitted, more in progress
 
- Allow all vTPM components to run in stub domains for increased security
 
- Allow all vTPM components to run in stub domains for increased security
- Update vtpm to 0.7.1 from 0.5.x
+
- Update vtpm to 0.7.4
  +
- Remove dom0-based vtpmd
  +
  +
* Guest EFI booting
  +
- status: tianocore in-tree, some build problems.
  +
prognosis: Poor.
  +
Needs new owner.
   
 
* libvirt/libxl integration (external)
 
* libvirt/libxl integration (external)
  +
- Update libvirt to 4.2
  +
status: Patch accepted
 
- Migration
 
- Migration
 
owner: cyliu@suse (?)
 
owner: cyliu@suse (?)
 
status: first draft implemented, not yet submitted
 
status: first draft implemented, not yet submitted
  +
prognosis: ?
 
- Itemize other things that need work
 
- Itemize other things that need work
 
To begin with, we need someone to go and make some lists:
 
To begin with, we need someone to go and make some lists:
Line 125: Line 156:
 
See http://libvirt.org/hvsupport.html
 
See http://libvirt.org/hvsupport.html
 
- Features available in xl/Xen but not available in libvirt/Xen
 
- Features available in xl/Xen but not available in libvirt/Xen
  +
  +
* Allow XSM to override IS_PRIV checks in the hypervisor
  +
owner: Daniel De Graaf
  +
status: patches against 4.3-unstable posted, awaiting approval
  +
prognosis: Good
  +
This makes it possible to allow some user domains limited access to
  +
dom0-only hypercalls. This could be used to allow a user-created
  +
toolstack domain to administer its own set of VMs instead of relying
  +
on dom0's toolstack.
   
 
* V4V: Inter-domain communication
 
* V4V: Inter-domain communication
 
owner (Xen): jean.guyader@citrix.com
 
owner (Xen): jean.guyader@citrix.com
 
status (Xen): patches submitted
 
status (Xen): patches submitted
  +
prognosis: ?
 
owner (Linux driver): stefano.panella@citrix
 
owner (Linux driver): stefano.panella@citrix
 
status (Linux driver): in progress
 
status (Linux driver): in progress
   
* Wait queues for mm (NEW)
+
* Wait queues for mm
 
owner: ?
 
owner: ?
 
status: Draft posted Feb 2012; more work to do.
 
status: Draft posted Feb 2012; more work to do.
  +
prognosis: Poor
   
* xl vm-{export,import}
 
owner: ?
 
status: ?
 
Allow xl to import and export VMs to other formats; particularly
 
ovf, perhaps the XenServer format, or more.
 
 
 
* xl PVUSB pass-through for both PV and HVM guests
 
* xl PVUSB pass-through for both PV and HVM guests
owner: ?
+
owner: George
 
status: ?
 
status: ?
  +
prognosis: Fair
 
xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both PV and HVM guests).
 
xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both PV and HVM guests).
 
- port the xm/xend functionality to xl.
 
- port the xm/xend functionality to xl.
Line 152: Line 189:
   
 
* xl USB pass-through for HVM guests using Qemu USB emulation
 
* xl USB pass-through for HVM guests using Qemu USB emulation
owner: ?
+
owner: George
status: ?
+
status: Config file pass-through submitted.
  +
prognosis: Good
 
xm/xend with qemu-traditional supports USB passthrough to HVM guests using the Qemu emulated USB controller.
 
xm/xend with qemu-traditional supports USB passthrough to HVM guests using the Qemu emulated USB controller.
 
The HVM guest does not need any special drivers for this feature.
 
The HVM guest does not need any special drivers for this feature.
Line 163: Line 201:
 
* xl QXL Spice support
 
* xl QXL Spice support
 
owner: Zhou Peng
 
owner: Zhou Peng
  +
prognosis: Fair
status: Patches against 4.2-unstable posted, need refresh & resubmit
 
  +
status: Patches against 4.3-unstable posted, awaiting approval
  +
  +
* Remove hardcoded mobprobe's in xencommons
  +
owner: ?
  +
status: ?
  +
prognosis: Poor.
   
 
* openvswitch toostack integration
 
* openvswitch toostack integration
 
owner: roger@citrix
 
owner: roger@citrix
  +
prognosis: ?
 
status: Sample script posted by Bastian ("[RFC] openvswitch support script")
 
status: Sample script posted by Bastian ("[RFC] openvswitch support script")
   
 
* Rationalized backend scripts (incl. driver domains)
 
* Rationalized backend scripts (incl. driver domains)
 
owner: roger@citrix
 
owner: roger@citrix
  +
status: libxl hotplug sumbmitted. Protocol still needs to be finalized.
status: ?
 
  +
prognosis: Fair
  +
  +
* Xen EFI features
  +
- dom0 able to make use of EFI run-time services
  +
- Xen able to use EFI boot-time services (?)
  +
- Xen able to detect the existence of a signed Linux binary, and leave EFI boot-time services enabled for dom0 to use when appropriate
  +
- dom0 able to use boot-time EFI services and disable them when done
  +
status: No owner.
  +
prognosis: Probably not for 4.4
   
 
* Serial console improvements
 
* Serial console improvements
 
owner: ?
 
owner: ?
 
status: Stalled (see below)
 
status: Stalled (see below)
  +
prognosis: Probably not for 4.4.
 
-xHCI debug port (Needs hardware)
 
-xHCI debug port (Needs hardware)
 
-Firewire (needs hardware)
 
-Firewire (needs hardware)
 
* Remove hardcoded mobprobe's in xencommons
 
owner: ?
 
status: ?
 
   
 
* Make storage migration possible
 
* Make storage migration possible
 
owner: ?
 
owner: ?
status: ?
+
status: none
  +
prognosis: Probably delay until 4.4
 
There needs to be a way, either via command-line or via some hooks,
 
There needs to be a way, either via command-line or via some hooks,
 
that someone can build a "storage migration" feature on top of libxl
 
that someone can build a "storage migration" feature on top of libxl
Line 192: Line 244:
 
* Full-VM snapshotting
 
* Full-VM snapshotting
 
owner: ?
 
owner: ?
status: ?
+
status: none
  +
prognosis: Probably delay until 4.4
 
Have a way of coordinating the taking and restoring of VM memory and
 
Have a way of coordinating the taking and restoring of VM memory and
 
disk snapshots. This would involve some investigation into the best
 
disk snapshots. This would involve some investigation into the best
Line 199: Line 252:
 
* VM Cloning
 
* VM Cloning
 
owner: ?
 
owner: ?
status: May need review
+
status: none
  +
prognosis: Probably need 4.4
 
Again, a way of coordinating the memory and disk aspects. Research
 
Again, a way of coordinating the memory and disk aspects. Research
 
into the best way to do this would probably go along with the
 
into the best way to do this would probably go along with the
 
snapshotting feature.
 
snapshotting feature.
   
  +
* xl vm-{export,import}
  +
owner: ?
  +
status: none
  +
prognosis: Prob put off until 4.4 (or GSoC project)
  +
Allow xl to import and export VMs to other formats; particularly
  +
ovf, perhaps the XenServer format, or more.
  +
 
* Memory: Replace PoD with paging mechanism
 
* Memory: Replace PoD with paging mechanism
 
owner: george@citrix
 
owner: george@citrix
status: May need review
+
status: none
  +
prognosis: Prob put off until 4.4
   
 
* PV audio (audio for stubdom qemu)
 
* PV audio (audio for stubdom qemu)
 
owner: stefano.panella@citrix
 
owner: stefano.panella@citrix
 
status: ?
 
status: ?
  +
prognosis: ?
   
 
* IllumOS support
 
* IllumOS support
 
owner: Igor Kozhukov
 
owner: Igor Kozhukov
status: In progress
+
status: Stopped
 
* Managed domains?
 
 
</pre>
 
</pre>
   

Revision as of 12:14, 17 January 2013

Proposed timeline

We are planning on a 9-month release cycle. Xen 4.2 was released on 17 September 2012. Based on that, below are our estimated dates:

  • Feature Freeze: 25 March, 2013
  • First RC: 6 May 2013
  • Release: 17 June 2013

The RCs and release will of course depend on stability and bugs, and will therefore be fairly unpredictable. The feature freeze may be slipped for especially important features which are near completion.

Feature list

Below is a list of features we're tracking for this release. If you have updates to a tracked feature you are working on, or are working on a feature and wish it to be tracked, please let me know.

NB: Several of the items on this list marked (external). These are not part of the Xen tree, but are directly related to our users' experience (e.g., work in Linux or qemu) or to integration with other important projects (e.g., libvirt indings). Since all of these are part of the Xen community work, and comes from the same pool of labor, it makes sense to track the progress here, even though they won't explicitly be released as part of 4.3.

Meanings of prognoses:

  • Excellent: It would be very unlikely for this not to be finished in time.
  • Good: Everything is on track, and is likely to make it.
  • Fair: A pretty good chance of making it, but not as certain
  • Poor: Likely not to make it unless intervention is made
  • Not for 4.3: Self-explanatory


Completed

* Serial console improvements
  -EHCI debug port

* Default to QEMU upstream (partial)
 - pci pass-thru (external)
 - enable dirtybit tracking during migration (external)
 - xl cd-{insert,eject} (external)

* CPUID-based idle (don't rely on ACPI info f/ dom0)

* Persistent grants for blk (external)
 - Linux

Bugs

* xl, compat mode, and older kernels
  owner: ?
  Many older 32-bit PV kernels that can run on a 64-bit hypervisor with
  xend do not work when started with xl.  The following work-around seems to
  work:
    xl create -p lightning.cfg
    xenstore-write /local/domain/$(xl domid lightning)/device/vbd/51713/protocol x86_32-abi
    xl unpause lightning
  This node is normally written by the guest kernel, but for older kernels
  seems not to be.  xend must have a work-around; port this work-around to xl.


Not yet completed

* PVH mode (w/ Linux)
  owner: mukesh@oracle
  status (Linux): 3rd draft patches posted.  
  status (Xen): RFC submitted
  prognosis: Good

* Event channel scalability
  owner: wei@citrix
  status: RFC submitted
  prognosis: Good
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* ARM v7 server port
  owner: ijc@citrix
  prognosis: Excellent
  status: Core hypervisor and Linux patches accepted.  Tools patches submitted.

* ARM v8 server port (tech preview)
  owner: ijc@citrix
  status: ?
  prognosis: Tech preview only

* NUMA scheduler affinity
  critical
  owner: dario@citrix
  status: Patches posted
  prognosis: Excellent

* NUMA Memory migration 
  owner: dario@citrix
  status: in progress
  prognosis: Fair

* blktap3
  owner: thanos@citrix
  status: RFCs posted
  prognosis: Not for 4.3

* Default to QEMU upstream
 > Add "intel-hda" to xmexample file, since it works with 64-bit Win7/8
 - qemu-based stubdom (Linux or BSD libc)
   owner: anthony@citrix
   status: in progress
   prognosis: ?
   qemu-upstream needs a more fully-featured libc than exists in
   mini-os.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

* Persistent grants for blk (external)
  owner: roger@citrix
  status: qemu patches submitted
  prognosis: Good

* Persistent grants for net
  owner: annie.li@citrix
  status: Initial implementation posted
  prognosis: ?

* Multi-page blk rings (external)
 - blkback in kernel roger@citrix
 - qemu blkback
  status: Starting soon
  prognosis: Fair

* Multi-page net protocol (external)
  owner: ijc@citrix or annie.li@oracle
  status: Initial patches posted (by Wei Liu)
  expand the network ring protocol to allow multiple pages for
  increased throughput

* Scalability: 16TiB of RAM
  owner: jan@suse
  status: In progress
  prognosis: Good

* vTPM updates
  owner: Matthew Fioravante @ Johns Hopkins
  prognosis: Good
  status: some patches submitted, more in progress
  - Allow all vTPM components to run in stub domains for increased security
  - Update vtpm to 0.7.4
  - Remove dom0-based vtpmd

* Guest EFI booting
 - status: tianocore in-tree, some build problems.
   prognosis: Poor.
   Needs new owner.

* libvirt/libxl integration (external)
 - Update libvirt to 4.2
   status: Patch accepted
 - Migration
   owner: cyliu@suse (?)
   status: first draft implemented, not yet submitted
   prognosis: ?
 - Itemize other things that need work
   To begin with, we need someone to go and make some lists:
   - Features available in libvirt/KVM not available in libvirt/libxl
     See http://libvirt.org/hvsupport.html
   - Features available in xl/Xen but not available in libvirt/Xen

* Allow XSM to override IS_PRIV checks in the hypervisor
  owner: Daniel De Graaf
  status: patches against 4.3-unstable posted, awaiting approval
  prognosis: Good
  This makes it possible to allow some user domains limited access to
  dom0-only hypercalls. This could be used to allow a user-created
  toolstack domain to administer its own set of VMs instead of relying
  on dom0's toolstack.

* V4V: Inter-domain communication
  owner (Xen): jean.guyader@citrix.com
  status (Xen): patches submitted
  prognosis: ?
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

* Wait queues for mm
  owner: ?
  status: Draft posted Feb 2012; more work to do.
  prognosis: Poor

* xl PVUSB pass-through for both PV and HVM guests
  owner: George
  status: ?
  prognosis: Fair
  xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both PV and HVM guests).
  - port the xm/xend functionality to xl.
  - this PVUSB feature does not require support or emulation from Qemu.
  - upstream the Linux frontend/backend drivers. Current work-in-progress versions are in Konrad's git tree.
  - James Harper's GPLPV drivers for Windows include PVUSB frontend drivers.

* xl USB pass-through for HVM guests using Qemu USB emulation
  owner: George
  status: Config file pass-through submitted.
  prognosis: Good
  xm/xend with qemu-traditional supports USB passthrough to HVM guests using the Qemu emulated USB controller.
  The HVM guest does not need any special drivers for this feature.
  So basicly the qemu cmdline needs to have:
     -usb -usbdevice host:xxxx:yyyy
  - port the xm/xend functionality to xl.
  - make sure USB passthrough with xl works with both qemu-traditional and qemu-upstream.

* xl QXL Spice support
  owner: Zhou Peng
  prognosis: Fair
  status: Patches against 4.3-unstable posted, awaiting approval

* Remove hardcoded mobprobe's in xencommons
  owner: ?
  status: ?
  prognosis: Poor.

* openvswitch toostack integration
  owner: roger@citrix
  prognosis: ?
  status: Sample script posted by Bastian ("[RFC] openvswitch support script")

* Rationalized backend scripts (incl. driver domains)
  owner: roger@citrix
  status: libxl hotplug sumbmitted.  Protocol still needs to be finalized.
  prognosis: Fair

* Xen EFI features
 - dom0 able to make use of EFI run-time services
 - Xen able to use EFI boot-time services (?)
 - Xen able to detect the existence of a signed Linux binary, and leave EFI boot-time services enabled for dom0 to use when appropriate
 - dom0 able to use boot-time EFI services and disable them when done 
 status: No owner.
 prognosis: Probably not for 4.4

* Serial console improvements
  owner: ?
  status: Stalled (see below)
  prognosis: Probably not for 4.4.
  -xHCI debug port (Needs hardware)
  -Firewire (needs hardware)

* Make storage migration possible
  owner: ?
  status: none
  prognosis: Probably delay until 4.4
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: none
  prognosis: Probably delay until 4.4
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: none
  prognosis: Probably need 4.4
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* xl vm-{export,import}
  owner: ?
  status: none
  prognosis: Prob put off until 4.4 (or GSoC project)
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.
  
* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: none
  prognosis: Prob put off until 4.4

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?
  prognosis: ?

* IllumOS support
  owner: Igor Kozhukov
  status: Stopped