Engineering deep dive · Compute

More vCPUs can make a virtual machine slower.

A vCPU is scheduled time, not a permanently owned core. On a multi-socket host, where its memory lives can matter as much as how many processors the VM sees.

The counterintuitive resultA VM that spans two NUMA nodes may wait longer for scheduling and fetch more remote memory than a smaller VM contained inside one node.

The physical topology

Two sockets are not one uniform pool.

Each processor socket is close to its own memory channels. A core can access memory attached to the other socket, but that request crosses the interconnect and usually costs more latency and bandwidth.

One memory load

Guest addresses pass through two translations.

Guest virtualguest page tablesGuest physicalEPT / NPTHost physicalNUMA placementLocal or remote DRAM

TLBs cache translations. A miss can require a page walk; an EPT/NPT fault may cause additional hypervisor work. Huge pages can reduce translation pressure, but they also make allocation and migration less flexible.

Inspect the host

Do not tune topology you have not measured.

lscpu -e=CPU,NODE,SOCKET,CORE,ONLINE
numactl --hardware
numastat -m
virsh vcpupin <vm>
virsh numatune <vm>

lscpu reveals CPU-to-node placement; numactl shows node capacity and distances; numastat exposes local versus remote allocation.

Illustrative measurement notebook

PlacementMedian memory latencyMeaning
CPU and memory on node 095 nsLocal path
CPU on node 0, memory on node 1145 nsRemote path
VM spread without policyVariableScheduler and allocator decide

These are explanatory numbers, not a VMBook benchmark. Publish the host model, firmware, kernel, command and raw result before comparing systems.

When each control helps—and what it costs

Right-size vCPUs first

Extra idle vCPUs still enlarge the scheduling problem. Start with the workload’s runnable demand and increase only after measuring CPU ready or steal time.

Pinning

Pinning can reduce movement and improve repeatability, but reserved CPUs reduce scheduler flexibility and can strand capacity.

Virtual NUMA

Large guests may benefit when their topology reflects the host. The guest OS and application must also be NUMA-aware.

Huge pages

They reduce TLB pressure for some workloads, but require contiguous memory and complicate overcommit, reclaim and migration.