Local AI PC Build Guide: VRAM, RAM, Storage, and Power

A local-AI PC can mean interactive language-model inference, image generation, embeddings for retrieval, experimentation with fine-tuning, or a machine that serves several users. Those jobs can stress memory capacity, memory bandwidth, compute, storage, CPU resources, and software support in different proportions. Define the models, precision or quantization, context target, concurrency, operating system, and applications first; only then turn those requirements into hardware constraints.

Software compatibility is part of the hardware specification. PyTorch currently publishes separate installation paths for CUDA and ROCm acceleration, while Ollama documents NVIDIA, AMD ROCm, Metal, and Vulkan support with platform- and GPU-specific conditions. A GPU can have attractive theoretical compute or memory specifications and still be the wrong purchase if the framework, model format, kernel, driver, or application you need does not support it well.

For GPU-accelerated local inference, start by checking whether the exact application supports the GPU and operating system, then inspect memory demand. Ollama, for example, currently documents supported NVIDIA GPUs by compute capability, separate AMD ROCm support lists for Linux and Windows, Metal acceleration on Apple hardware, and additional Windows/Linux GPU support through Vulkan. PyTorch likewise distinguishes CUDA and ROCm builds. Do not replace those compatibility checks with a vendor logo or a raw TOPS figure.

VRAM capacity determines how much model and runtime state can remain local to the GPU, but capacity alone does not predict speed. Architecture, memory bandwidth, kernels, quantization format, model structure, context, batch size, and the amount of CPU offload all matter. Ollama explicitly recommends checking its PROCESSOR field to see the GPU/CPU split, which is more useful than assuming a model that launches is fully GPU-resident.

A rough weight-only calculation is parameter count multiplied by storage precision: one billion parameters at 16 bits is about 2 GB in decimal units before any runtime overhead, while an idealized 8-bit representation is about 1 GB. Real quantization formats can add metadata, scales, higher-precision components, padding, or other implementation overhead, so that arithmetic is a planning sanity check rather than an exact VRAM-fit promise.

Runtime memory also includes items that a weight-file size does not describe. For language models, context creates key/value cache and other working allocations; training adds substantially different state. Hugging Face documents that 8-bit bitsandbytes quantization can halve model memory usage relative to the corresponding higher-precision load, while also exposing a memory-footprint function precisely because actual loaded memory is the relevant quantity. Treat each framework and model as measurable rather than converting parameter count into a universal GPU recommendation.

Long-context chat, coding agents, document analysis, and retrieval workflows can require much more runtime memory than a short prompt. Ollama states directly that increasing context length increases the memory required to run a model. Its current defaults also vary by available VRAM, which illustrates why a simple “this model fits this card” statement can omit an important operating condition.

Choose a realistic context target before sizing memory. If the intended workflow needs long context, test or obtain measurements at that context and with the same quantization and backend. Do not assume the maximum context advertised by a model can be used at the same memory footprint or speed as a short interactive session.

System memory must cover the operating system, AI application, model-loading path, CPU-side model data or offloaded layers, retrieval services, browsers, development tools, and any other processes used alongside inference. A GPU with enough VRAM for one model does not establish the correct system-RAM capacity for the complete workflow.

CPU offload can let a workload operate when all desired model state does not reside on the GPU, but it changes where data lives and can change performance substantially. Size RAM from observed or documented peak use for the chosen software and leave practical headroom for concurrent services. Do not treat RAM as a transparent substitute for VRAM or quote a fixed VRAM-to-RAM ratio across different runtimes.

Local AI can accumulate several quantizations or versions of the same model, image-generation checkpoints, adapters, caches, datasets, vector indexes, container images, Python environments, and generated output. Plan SSD capacity from the files you expect to retain rather than from the size of one downloaded model. Keep backup capacity separate from working storage.

Storage speed matters most when models and datasets are loaded, copied, indexed, or streamed; once a workload is resident in memory, faster SSD sequential throughput does not automatically increase token generation or GPU compute performance. Estimate capacity first, then use measurements from the actual loading and data path before paying for storage performance that the workload cannot use.

A high-power accelerator can determine the PSU, case, cooling, and connector requirements of the whole machine. Use the exact card vendor specification rather than a generic GPU-family assumption. As one scale example, NVIDIA specifies 575 W total graphics power and 1000 W required system power for its RTX 5090 Founders Edition reference configuration, while explicitly noting that system requirements vary with configuration and add-in-card specifications vary by manufacturer. That is an example of why exact-part validation matters, not a recommendation to buy that card or PSU size.

For multiple GPUs, check motherboard slot spacing, electrical lane allocation, CPU/chipset topology, PSU connectors and capacity, chassis clearance, airflow, and whether the software can actually distribute the chosen workload across devices. Multiple visible GPUs do not guarantee pooled VRAM or useful scaling. Confirm the runtime’s multi-device behavior before building an expensive platform around an assumed aggregate memory pool.

First, name the exact applications and model families. Record the quantization or precision, realistic context length, expected concurrency, and whether the job is inference, training or fine-tuning, image/video generation, embeddings, or a mixed workflow. Second, verify the GPU backend and driver support on the operating system you intend to use. Third, measure or source credible memory use for that exact workload instead of relying on parameter count alone.

Then choose enough GPU memory and system RAM for that envelope with reasonable headroom, estimate the model and dataset library for storage, and validate the complete platform: CPU needs, PCIe topology, motherboard expansion, PSU and connectors, case fit, cooling, networking, and backup. Re-run the sizing exercise when the model, context, framework, or concurrency target changes. A local-AI workstation is a platform for a defined software workload, not a universal ranked parts list.