Session

Journey of advancing device migration for virtio PCI hardware devices

Speakers

Parav Pandit
Yishai Hadas
Avihai Horon
Feng Liu
Satananda Burla

Label

Nuts and Bolts

Session Type

Talk

Contents

Description

Virtio network devices are fundamental to cloud VMs, evolving from para-virtual devices to vDPA and hardware PCI devices, including both physical and virtual functions. These PCI virtual functions are commonly attached as pass-through devices using the VFIO subsystem to VMs. Live migration of VMs with pass-through virtual functions is essential for hypervisor infrastructure. This talk details our journey through various design aspects, implementation challenges, their solutions, and performance benchmarks.

We highlight specific design considerations that influenced the specification draft and its implementation. We also explore the key differences between our approach and both a vdpa vendor-based approach and briefly, the IDPF. This technical paper presents, for the first time globally, performance metrics for virtio PCI devices using IOMMU-based dirty page tracking. We discuss performance benchmark results from single to multiple devices and how the pre-copy approach further reduces migration downtime. Finally, we review the current progress of the Virtio specification developed by the OASIS open standards community. We conclude this talk by sharing lessons learned from developing code and specifications concurrently, closing gaps, and discussing the trade-offs of various approaches.