Difference between revisions of "Linux stub domains"

From Xen
(diagram for discussion)
 
(6 intermediate revisions by one other user not shown)
Line 22: Line 22:
   
 
* Jul 2018, [https://lists.gt.net/xen/devel/523881 v1 patch]
 
* Jul 2018, [https://lists.gt.net/xen/devel/523881 v1 patch]
  +
  +
== Interfaces between stubdom QEMU and dom0 ==
  +
  +
* (June 2019) [https://groups.google.com/forum/m/#!topic/qubes-devel/ga1ToCnXz14 qemu-stubdom<=>dom0 interface changes]
  +
* (June 2019) [https://groups.google.com/forum/m/#!topic/qubes-devel/YrteVv_VyDw Requirements for PVH stubdoms]
   
 
== Xen Summit 2019 Design Session ==
 
== Xen Summit 2019 Design Session ==
   
''(diagram for architecture proposed by Brendan Kerrigan)''
+
=== Display architecture proposed by Brendan K ===
  +
  +
* migrate from display changer on Qubes and OpenXT surfman, to upstream Xen interfaces (from EPAM)
  +
* use Linux 5.1 drm-front driver for displayif
  +
* share common codebase for: EPAM, Qubes, OpenXT, Redfield
  +
  +
[[File:2019-07-common-linux-stubdom.png]]
  +
  +
  +
=== QMP discussion ===
  +
  +
Proposal:
  +
  +
- LibXL and block backend: change "xenstore stuff" to "QMP stuff"
  +
  +
Concerns:
  +
  +
# QMP is not a simple protocol.
  +
# xenstore-based protocol is much simpler than a JSON-encapsulated protocol.
  +
# There have been multiple historical bugs in parsing of JSON in C.
  +
# If we extend QMP support in LibXL and are communicating with a ''potentially compromised'' QEMU process, it would be a security regression.
  +
  +
Possible mitigations for a ''potentially compromised QEMU''
  +
  +
# Reduce privileges given to the QEMU instance
  +
# Limit interfaces to the QEMU instance
  +
# Work with upstream QEMU to propose a new safer-than-JSON alternative marshalling format backend
  +
# Minimize what LibXL does with QMP messages: use an off-the-shelf JSON firewall/translator/filter in the untrusted guest, limited to the QEMU subset of JSON
  +
# Don't talk to QEMU after the device is started: mostly done today, but does not work for some corner cases: optical drive insert, device hotplug
  +
# Have an option to intentionally break all QMP communications
  +
  +
Other notes:
  +
  +
* Qubes uses dracut for build, OpenXT uses OE
  +
* For OpenXT/Qubes, many QEMU functions can build-time disabled
  +
* For Debian/SuSE distro, the generic QEMU binary has many functions enabled. It may be an option to ship "minimal QEMU" and "minimal kernel" configured binaries for distro stubdoms.
  +
  +
Actions:
  +
  +
* Marek to resubmit Linux stubdom patches to xen-devel, ask people to comment on which patches have consensus or open discussion items
  +
* Ian J can review patches in mid-August
  +
* Upstream Xen will merge the ones which are not contended
  +
* Work towards consensus on contended patches
  +
  +
== Additional Notes ==
  +
  +
=== QMP in Qubes and OpenXT ===
  +
  +
Connection between the qemu device model and the libxl toolstack requires a communication channel to be established between the separate domains where they are run, which is then used to deliver QMP traffic in both Qubes and OpenXT.
  +
  +
Qubes uses a vchan-socket-proxy:
  +
  +
[https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/rootfs/init#L147 https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/rootfs/init#L147]
  +
  +
[https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/helpers/vchan-socket-proxy.c https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/helpers/vchan-socket-proxy.c]
  +
  +
OpenXT uses a qemu character device that delivers via Argo to a helper process in dom0:
  +
  +
[https://github.com/OpenXT/xenclient-oe/blob/9.0.2/recipes-extended/qemu-dm/qemu-dm/qmp-argo-char-driver.patch https://github.com/OpenXT/xenclient-oe/blob/9.0.2/recipes-extended/qemu-dm/qemu-dm/qmp-argo-char-driver.patch]
  +
  +
[https://github.com/OpenXT/xctools/blob/9.0.2/qmp_helper/src/qmp_helper.c https://github.com/OpenXT/xctools/blob/9.0.2/qmp_helper/src/qmp_helper.c]
  +
  +
The Xen toolstack libxl is currently dropping QMP Events after parsing them from JSON, so filtering at the qemu source could reduce the volume of interactions over the channel:
   
  +
[https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/libxl_qmp.c;h=9aabad74fabd7bfd27b2bac153973c612a46a3c2;hb=HEAD#l278 https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/libxl_qmp.c;h=9aabad74fabd7bfd27b2bac153973c612a46a3c2;hb=HEAD#l278]
+-----------------------------------------------------+
 
| +-------------------------------------------------+ |
 
| | | |
 
| | | |
 
| | | |
 
| | +-------+ +---------+ +---------+ +-----+ | |
 
| | |DRM DCL| | ALSA | | evdev | | qmp | | |
 
| | +-------+ +---------+ +---------+ QEmu +-----+ | |
 
| +-------------------------------------------------+ |
 
| | | | | |
 
| | | | | |
 
| +---------+ +---------+ +---------+ +-----------+ |
 
| | card0 | | ALSA/PA | | evdev | |argo_stream| |
 
| +---------+ +---------+ +---------+ +-----------+ |
 
+-----------------------------------------------------+
 
| +---------+ +---------+ +---------+ +------+ |
 
| | Xen DRM | | Xen Snd | | Xen KBD | Linux | Xen | |
 
| | Front | | Front | | Front | Stubdom| Argo | |
 
| +---------+ +---------+ +---------+ +------+ |
 
+-----------------------------------------------------+
 

Latest revision as of 05:09, 16 October 2019

Linux stub domains are used in QubesOS and OpenXT.

Upstreaming

todo: add remaining patches

00/17 Add support for qemu-xen runnning in a Linux-based stubdomain:

05/17 libxl: Handle Linux stubdomain specific QEMU options:

08/17 xl: add stubdomain related options to xl config parser

Interfaces between stubdom QEMU and dom0

Xen Summit 2019 Design Session

Display architecture proposed by Brendan K

  • migrate from display changer on Qubes and OpenXT surfman, to upstream Xen interfaces (from EPAM)
  • use Linux 5.1 drm-front driver for displayif
  • share common codebase for: EPAM, Qubes, OpenXT, Redfield

2019-07-common-linux-stubdom.png


QMP discussion

Proposal:

- LibXL and block backend: change "xenstore stuff" to "QMP stuff"

Concerns:

  1. QMP is not a simple protocol.
  2. xenstore-based protocol is much simpler than a JSON-encapsulated protocol.
  3. There have been multiple historical bugs in parsing of JSON in C.
  4. If we extend QMP support in LibXL and are communicating with a potentially compromised QEMU process, it would be a security regression.

Possible mitigations for a potentially compromised QEMU

  1. Reduce privileges given to the QEMU instance
  2. Limit interfaces to the QEMU instance
  3. Work with upstream QEMU to propose a new safer-than-JSON alternative marshalling format backend
  4. Minimize what LibXL does with QMP messages: use an off-the-shelf JSON firewall/translator/filter in the untrusted guest, limited to the QEMU subset of JSON
  5. Don't talk to QEMU after the device is started: mostly done today, but does not work for some corner cases: optical drive insert, device hotplug
  6. Have an option to intentionally break all QMP communications

Other notes:

  • Qubes uses dracut for build, OpenXT uses OE
  • For OpenXT/Qubes, many QEMU functions can build-time disabled
  • For Debian/SuSE distro, the generic QEMU binary has many functions enabled. It may be an option to ship "minimal QEMU" and "minimal kernel" configured binaries for distro stubdoms.

Actions:

  • Marek to resubmit Linux stubdom patches to xen-devel, ask people to comment on which patches have consensus or open discussion items
  • Ian J can review patches in mid-August
  • Upstream Xen will merge the ones which are not contended
  • Work towards consensus on contended patches

Additional Notes

QMP in Qubes and OpenXT

Connection between the qemu device model and the libxl toolstack requires a communication channel to be established between the separate domains where they are run, which is then used to deliver QMP traffic in both Qubes and OpenXT.

Qubes uses a vchan-socket-proxy:

https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/rootfs/init#L147

https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/helpers/vchan-socket-proxy.c

OpenXT uses a qemu character device that delivers via Argo to a helper process in dom0:

https://github.com/OpenXT/xenclient-oe/blob/9.0.2/recipes-extended/qemu-dm/qemu-dm/qmp-argo-char-driver.patch

https://github.com/OpenXT/xctools/blob/9.0.2/qmp_helper/src/qmp_helper.c

The Xen toolstack libxl is currently dropping QMP Events after parsing them from JSON, so filtering at the qemu source could reduce the volume of interactions over the channel:

https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/libxl_qmp.c;h=9aabad74fabd7bfd27b2bac153973c612a46a3c2;hb=HEAD#l278