Virtualization

Overview

Virtualization lets one physical server present itself as several independent machines. A software layer called a hypervisor sits between the hardware and the workloads, partitioning the machine's resources and giving each tenant the impression of having their own dedicated system. It is the foundation of conventional cloud computing, because it allows providers to share expensive hardware efficiently among many customers at once.

How it works

The hypervisor schedules access to the underlying CPU, memory, and devices among the virtual machines running on top of it, translating and isolating their requests so they cannot interfere with one another. This flexibility is valuable, but it has a cost: the hypervisor consumes some cycles for itself and introduces variability in timing, since each virtual machine is contending, ultimately, for shared physical resources.

Why it matters

For most general workloads, virtualization's overhead is a worthwhile trade for its flexibility. For demanding AI training, it is often not. These workloads press against the limits of memory bandwidth and interconnect speed, where even a few percent of overhead translates into longer runs and higher cost on expensive hardware, and where the timing variability of shared resources undermines predictability. This is the case for bare metal, where the absence of a hypervisor returns the machine's full, consistent performance to a single tenant.