Developer Guide

There are still TODOs and things to improve to make virtio-mem support even more use cases.

Linux guest driver TODOs

1. Support the memmap_on_memory kernel command line paramater for memory hotplugged via virtio-mem.
2. alloc_contig_range() improvements
3. Defragmentation

Linux host TODOs

1. Reclaiming empty page tables
2. Optimizing for sparse memory mappings

QEMU TODOs

1. iothread=X option for virtio-mem devices**: Process guest requests (plug/unplug memory blocks), inluding discarding memory, preallcoating memory and updating VFIO mappings, via a separate thread; avoid holding the BQL (Big QEMU Lock) for a long time during expensive operations.
2. Protection of unplugged memory: catch access to logically unplugged memory and handle it accordingly (e.g., inject interrupt into VM).

kdump/kexec-tools Handling

virito-mem adds System RAM (virtio_mem) - resources as child resources under the corresponding virtio device resource. This is visible via /proc/iomem to user space. For example:

00000000-00000fff : Reserved
00001000-0009fbff : System RAM
[...]
fffc0000-ffffffff : Reserved
100000000-13fffffff : System RAM
140000000-33fffffff : virtio0
  140000000-147ffffff : System RAM (virtio_mem)
340000000-53fffffff : virtio1
  340000000-34fffffff : System RAM (virtio_mem)
540000000-5bfffffff : PCI Bus 0000:00

kexec-tools must not use this memory for placing kexec images, or use this memory when building the initial memory map (e.g., e820 map) for the kexec kernel. This is already done implicitly due to the way the resources show up as child resources of virtio devices.

However, kexec-tools should add this memory to the list of memory (via the elfcorehdr) to dump via kdump.

ACPI

In general, virtio-mem can live without ACPI. However, there has to be a way to expose the maximum possible pfn that can be used for hotplugged memory to the guest OS, for example, to properly setup swiotlb during boot in Linux. Currently, virtio-mem on x86-64 relies on ACPI SRAT tables to communicate the memory range that can be use for memory hotplug ("max_possible_pfn").

Future Work

• Indicate the guest status / guest errors via virtio-mem devices.
• OOM handling / early detection of low-memory situations.
• Guest-triggered shrinking of the usable region.