IOMMU Groups and PCIe ACS Explained for VFIO Passthrough

On Linux systems used for VFIO device passthrough, an IOMMU group is the smallest set of devices the platform can safely isolate from other groups. The IOMMU translates and restricts device DMA, but PCIe topology can limit how finely those devices can be separated. That is why two devices can appear in the same group even when they occupy different logical functions or connectors.

The practical rule is that VFIO treats the group as the unit of ownership. If one device in a group cannot be isolated from its peers, assigning only that device to an untrusted guest would not provide the DMA isolation that users usually expect from passthrough.

A passed-through device can initiate DMA, so the host needs a way to prevent it from reading or writing arbitrary system memory. The IOMMU provides that address-translation and permission boundary by mapping device-visible I/O virtual addresses to permitted physical memory.

But the IOMMU cannot protect a transaction it never sees. PCIe peer-to-peer paths can allow traffic between devices below a bridge or switch. Red Hat's virtualization documentation explains that if an interconnect can redirect a transaction before it reaches the IOMMU, those devices cannot automatically be considered isolated merely because the IOMMU exists.

PCI Express Access Control Services, usually abbreviated ACS, provide controls used to manage peer-to-peer transaction routing and isolation through PCIe components. In a passthrough topology, ACS capability at relevant root ports, switches, bridges and multifunction devices can determine whether downstream endpoints can be treated as separate isolation domains.

Red Hat documents ACS as the hardware mechanism used to maintain isolation within IOMMU grouping. If a point in the PCIe path lacks the required isolation capability, devices below that point may need to remain in one group because peer-to-peer DMA could bypass the IOMMU's intended boundary.

A discrete GPU often exposes more than one PCI function, such as the graphics function and an HDMI or DisplayPort audio function. Multifunction devices are part of the same physical endpoint and can have isolation relationships that differ from two completely separate add-in cards.

For VFIO, the important question is not whether the functions have different PCI addresses but whether the platform can isolate their DMA and peer traffic correctly. Linux kernel VFIO documentation therefore centers ownership on the IOMMU group rather than assuming every enumerated PCI function is independently assignable.

PCIe devices are arranged as a hierarchy. Endpoints may sit behind root ports, PCIe switches or legacy PCI bridges. Linux kernel documentation notes that topology can reduce isolation granularity; a PCIe-to-PCI bridge, for example, can hide distinctions among devices behind it from the IOMMU.

Red Hat also notes that a PCIe switch or bridge without appropriate ACS support can extend an IOMMU group. This is why moving a card to another motherboard slot can change grouping: the new slot may attach through a different root port or path with different isolation capabilities.

Desktop platforms commonly expose PCIe lanes from more than one place, including CPU root ports and chipset root ports. Those paths can have different ACS and IOMMU behavior. Two full-length slots that look similar on the board may therefore produce different passthrough isolation.

When a device is grouped with hardware you need to keep on the host, the least invasive diagnostic step is often to inspect the PCIe topology and, where practical, test another electrically suitable slot. That changes the hardware path rather than weakening the isolation policy.

Linux communities sometimes use kernel patches or parameters described as ACS overrides to split groups more aggressively. The crucial security distinction is that changing the kernel's grouping decision does not add missing transaction-control hardware to a root port, bridge, switch or endpoint.

If the physical topology still permits peer-to-peer traffic that bypasses the IOMMU, presenting smaller software groups can make the layout look more isolated than the hardware actually is. Do not treat an override as equivalent to vendor-documented native ACS when the threat model requires strong device isolation.

Start by identifying the target PCI device and its IOMMU group, then inspect every device in that group and the bridges or root ports above it. Linux exposes group membership through sysfs, while tools such as lspci help identify the PCI hierarchy and capabilities. Distribution virtualization tools can expose the same grouping through their device-management interfaces.

If the group contains unrelated hardware, compare the motherboard manual and platform documentation, test a different suitable slot if available, and check whether the intervening root port or switch documents ACS support. A different group after moving the card is a topology change; it is stronger evidence than simply forcing the kernel to report a different grouping.

Think of an IOMMU group as the smallest PCI device set that the system can hand to one ownership domain while preserving the isolation guarantees represented by the platform. The IOMMU controls DMA translation, PCIe ACS helps keep peer traffic on paths where those controls remain meaningful, and VFIO refuses to pretend that every PCI function is independently safe.

That model explains why passthrough guides care about CPU and firmware IOMMU support, root ports, switches, multifunction devices, ACS and slot choice at the same time. They are different pieces of one isolation problem, not interchangeable settings.