GPU Occupancy Explained: Warps, Waves, Registers, and Shared Memory

GPU occupancy is a resource-residency concept. NVIDIA defines CUDA kernel occupancy as the ratio of active warps on a Streaming Multiprocessor to the maximum active warps that the SM supports. AMD uses wavefront occupancy to reason about how many waves can be assigned to execution resources. In both cases, the useful question is how much schedulable work can remain resident at once, not whether every arithmetic lane is doing useful work every cycle.

That distinction matters because occupancy is not the same metric as GPU utilization, shader throughput, memory bandwidth utilization, frame rate, or execution-unit efficiency. A kernel can have high theoretical occupancy and still wait on dependencies or memory, while a deliberately lower-occupancy kernel can perform well when it uses its resident work efficiently.

Applications launch threads or work-items in blocks or workgroups. Hardware schedules those threads in architecture-defined groups: NVIDIA calls them warps, while AMD calls them wavefronts or waves. The existing Core Tech Tips shader-core guide explains those execution-group fundamentals; occupancy starts one layer later by asking how many such groups can simultaneously reside on the relevant SM, SIMD, compute unit, or related scheduling resource.

The terminology is not interchangeable across all vendors and generations. Current CUDA documentation defines a warp as 32 threads, while AMD architectures can use different wave modes. Occupancy calculations therefore have to use the actual target architecture and launch configuration rather than a universal warp-or-wave count.

Registers are fast storage used for values needed by executing threads. They are also finite. NVIDIA documents a register budget per SM and explains that resident blocks and warps depend partly on the registers required by a kernel. AMD similarly identifies VGPR demand as a major occupancy limiter because resources must be available for assigned waves.

Reducing register use is not automatically an optimization. A compiler can spill values when register pressure is constrained, moving data to a slower memory-backed path. The resulting increase in nominal occupancy can therefore trade away local register access and lose performance. The relevant goal is a good resource-and-throughput balance, not the smallest possible register count.

CUDA shared memory is partitioned among resident thread blocks on an SM. AMD Local Data Share, or LDS, serves the analogous software-managed on-chip sharing role for workgroups. If each block or workgroup reserves a large amount, fewer groups can fit concurrently even when register capacity and thread slots would otherwise allow more.

This is why occupancy tools take dynamic shared-memory use into account. NVIDIA exposes occupancy APIs that accept block size and dynamic shared-memory size, while AMD profiling guidance calls out LDS-limited occupancy directly. The exact capacities and allocation granularities vary by architecture, so a fixed kilobyte threshold should not be generalized across GPUs.

A larger block or workgroup contains more threads and therefore more warps or waves. It can also multiply per-thread register demand and reserve group-scoped shared memory. Hardware additionally limits resident blocks, threads, warps or waves. The binding constraint is whichever resource prevents another complete group from becoming resident first.

This creates stepwise behavior rather than a smooth rule that larger groups always increase occupancy. A launch configuration can leave unused resources because another full block or workgroup no longer fits. Conversely, very small groups can encounter resident-block or scheduling limits before consuming the available thread capacity. Architecture-aware occupancy calculators exist precisely because the interaction is multidimensional.

When one execution group cannot make progress—for example while waiting for a memory operation or an instruction dependency—a GPU scheduler may issue work from another ready group. Keeping multiple independent warps or waves resident can therefore provide alternative work and hide some latency instead of leaving execution resources idle.

Occupancy only creates the opportunity for that substitution. Resident work still needs useful independent instructions, and the workload may be limited somewhere else. NVIDIA documents sufficient occupancy as one way to hide latency; AMD likewise explains occupancy in terms of having other wavefronts available while one stalls. Neither makes maximum residency a guarantee of maximum throughput.

Chasing the highest possible occupancy can force harmful compromises. Restricting registers can cause spilling, shrinking tiles can reduce data reuse, and changing group size can alter synchronization or memory-access behavior. Compute kernels with substantial per-thread state can intentionally run at lower occupancy while extracting more work from each resident group.

NVIDIA’s own CUTLASS documentation provides a concrete architectural example: efficient matrix-multiplication kernels use substantial register and shared-memory storage and can have relatively low occupancy, then rely on software pipelining to overlap data movement and computation. The lesson is not that low occupancy is better; it is that occupancy is one constraint among several and must be interpreted alongside the workload.

Theoretical occupancy asks what residency the launch configuration and resource requirements permit. Compiler resource reports and occupancy calculators can estimate this before or without a full performance capture. NVIDIA exposes runtime occupancy APIs and profiler tooling for this purpose; AMD tools can expose wave occupancy and resource limiters.

Achieved behavior is a runtime question. Work distribution, dependencies, barriers, memory behavior, launch rate and the duration of individual groups can make observed occupancy vary over time. AMD GPUOpen explicitly discusses runtime occupancy limiters beyond the simple static register/LDS calculation. A static occupancy percentage is therefore a starting point for diagnosis, not a complete performance profile.

If profiling shows unexpectedly little resident work, identify the binding resource first: registers, shared memory or LDS, block/workgroup size, architectural resident-group limits, barriers, or another runtime constraint. Then test whether changing that constraint actually improves the metric that matters. A higher occupancy number without a throughput, latency, or frame-time improvement is not an optimization by itself.

Keep comparisons architecture-specific. Register-file size, allocation rules, shared-memory capacity, maximum resident groups, wave or warp limits and scheduler behavior can change between GPU generations—even within one vendor. Use the target GPU’s programming and profiling documentation rather than copying a block size, register target, or occupancy percentage from unrelated hardware.