PCIe ACS and IOMMU Groups Explained: Why VFIO Devices Get Grouped Together

Linux defines an IOMMU group as a set of devices that can be isolated from every device outside that group. VFIO uses the group as a unit of ownership because direct device access includes DMA: a passed-through device can read or write memory without the CPU copying each transfer.

The important word is isolatable. A platform may have an IOMMU capable of translating DMA per device, yet the PCIe topology can still allow transactions between devices below a bridge or inside a multifunction device to avoid the path where the IOMMU can enforce that isolation. Linux groups devices conservatively when the hardware cannot prove a finer boundary.

PCI Express Access Control Services, or ACS, provide controls over transaction routing and redirection in the PCIe hierarchy. This matters when two endpoints sit below the same downstream port or switch. Hardware can potentially route peer-to-peer traffic locally instead of sending it upstream through the root complex.

Red Hat's virtualization documentation describes ACS as the hardware standard used to maintain isolation within IOMMU groups. Without appropriate ACS support, software must account for the possibility that peer-to-peer DMA can occur outside IOMMU protection. The safe result is often a larger group rather than pretending two endpoints are independently isolated.

Group composition follows hardware isolation and topology, not the labels users give devices. A GPU can expose multiple PCI functions, such as graphics and an audio controller. Devices can also share a downstream bridge or switch whose routing capabilities do not provide the isolation Linux requires.

The Linux VFIO documentation explicitly notes that topology can reduce isolation granularity. A PCIe-to-PCI bridge can hide devices behind the bridge from the IOMMU's point of view, while other PCIe arrangements can permit peer transactions that never become independently visible upstream. That is why moving a card to another physical slot can sometimes change grouping: the slot may connect through a different root port or chipset path. It is not guaranteed, because motherboard wiring and platform capabilities decide the actual topology.

In the traditional VFIO group/container model, the group is the minimum ownership unit. The kernel documentation's example first identifies a device's group through its sysfs iommu_group link and then checks the other devices contained in that group.

When multiple ordinary endpoints share a group, the host cannot simply keep using one of them normally while granting unrestricted VFIO ownership of another and still claim group-level isolation. The exact handling depends on the devices and current VFIO interface, but the security model is deliberately stricter than treating each PCI address as automatically independent.

When ACS controls are available at the necessary points in the PCIe path, transactions can be constrained or redirected so the isolation boundary is visible to the IOMMU. That can allow devices below shared fabric to be placed into separate groups when the complete topology supports it.

ACS is not equivalent to the IOMMU itself. The IOMMU controls DMA translation and access to system memory; ACS addresses PCIe routing behavior that could otherwise let peer transactions bypass the point where that protection is effective. Both concepts therefore appear together in passthrough discussions, but they solve different parts of the isolation problem.

On Linux, a device's sysfs entry links to its IOMMU group. The VFIO documentation demonstrates reading /sys/bus/pci/devices/<BDF>/iommu_group and listing the devices inside that group. Pair that information with lspci output so PCI bus addresses can be mapped to the GPU, audio function, USB controller, network adapter, bridge, or other hardware involved.

Inspect the complete group before changing drivers. If the desired endpoint shares a group, inspect the upstream PCIe topology and motherboard slot wiring before assuming a kernel workaround is necessary. Another slot can have a different upstream path, and firmware settings can affect exposed virtualization capabilities, but neither is guaranteed to create a different secure group.

An inconvenient IOMMU group is not merely Linux being arbitrary. It is a conservative statement about the isolation boundary the kernel can establish from the IOMMU, PCIe topology and routing capabilities it sees. Native ACS can provide the controls needed for finer isolation; missing ACS can force devices into a broader boundary.

For a gaming VM or workstation passthrough build, choose motherboard slots and platform topology with isolation in mind rather than assuming every x16 or x4 slot can be assigned independently. For systems that depend on security boundaries between mutually untrusted guests or users, preserve native hardware isolation and avoid treating software-forced group splitting as a substitute for it.