Short answer: There is no single RAM number that guarantees a good local-AI experience. Memory needs depend on the model file, quantization, context length, whether inference uses system RAM or GPU memory, and the work you do alongside it. Check the model’s own requirements and test your actual workflow before buying a laptop for one headline spec.
Two machines can both advertise “AI-ready” hardware and feel very different with a local coding assistant. One may have an NPU built for a supported Windows feature; another may have a discrete GPU with more usable video memory; a third may rely on system RAM and run a smaller quantized model. The badge is not the same thing as a guarantee that your chosen model will fit or respond quickly.
What consumes memory?
| Resource | What uses it | Why it matters |
|---|---|---|
| Model weights | The parameter data loaded for inference | Larger or less-compressed models need more memory |
| Context and key/value cache | Conversation history, code files, and tokens the model can reference | Longer context can use substantially more memory |
| Runtime and temporary buffers | The inference engine, prompt processing, and intermediate operations | The model file size is not the full working-set requirement |
| Other apps | Browser tabs, IDE, build tools, video calls, and OS | They compete with the model for RAM and GPU memory |
| GPU memory | Model layers and inference state when offloaded to a GPU | A discrete GPU can be fast, but limited VRAM can force partial offload |
Model size is only the first estimate
A downloaded model’s file size gives a rough sense of the memory needed to store its weights, but inference also needs working space. Quantization can reduce the size of the weights, often trading some precision for a smaller footprint. The exact effect depends on the model and runtime. A model that barely fits may leave little room for a long prompt, another app, or a second model.
Context length matters because a 4,000-token question and a 64,000-token repository are not the same workload. A coding tool that reads files, keeps multiple turns, and asks the model to reason over a large diff may need far more memory than a short offline summary. Ollama’s current coding guidance, for example, recommends a 64,000-token context for particular integrations and lists a specific model configuration requiring about 23 GB of VRAM. That is an example for those model settings—not a universal “local AI minimum.”
System RAM, VRAM, and unified memory are different
On a desktop with a discrete GPU, the graphics card has its own VRAM. If the model and its context do not fit there, the runtime may offload some work to system RAM or CPU, which can affect speed. More VRAM can help a GPU-hosted model fit, but does not guarantee a faster result if the model or runtime is not configured to use it.
On a laptop with integrated graphics or an Apple-silicon-style unified-memory design, the CPU and GPU share a pool. The model can use system memory that would otherwise serve the OS and applications. This can make a larger model fit than a small dedicated VRAM figure suggests, but it does not create unlimited bandwidth or memory. A browser, IDE, and large repository still need space.
An NPU is another processor for supported AI workloads. Microsoft’s Windows documentation says NPU acceleration is available when software targets it; the hardware’s TOPS rating does not tell you that every open model or desktop chatbot uses that NPU. Check the exact app’s supported execution provider and model requirements.
A practical planning table
Use these as workload categories, not rigid purchase specifications. Actual fit varies by model, quantization, runtime, and context.
| What you want to do | What to verify before buying | Common trade-off |
|---|---|---|
| Try small local models for short questions | Model’s published download size, system RAM left after OS/apps, CPU support | Lower cost, but responses may be slower or less capable |
| Use a coding assistant on a small project | IDE + runtime + model working set; realistic context length | More RAM helps multitasking; model quality still matters |
| Review a large repository locally | Context requirement, VRAM/unified memory, token throughput, file indexing | Long context and parallel tools can increase memory and latency |
| Run image or multimodal models | Model-specific GPU/NPU support and image/video memory needs | AI-capable label alone does not guarantee app compatibility |
| Keep normal work responsive while a model runs | Headroom for browser, IDE, calls, and background processes | A model that technically fits may still make the whole laptop feel cramped |
How to test before spending money
- Choose the actual task. “Local AI” is too broad. Decide whether you need code completion, repository Q&A, document summarisation, transcription, image generation, or something else.
- Find the exact model and context requirement. Read the runtime or vendor page for the quantization, context length, and supported hardware you plan to use.
- Check current machine memory. Close nothing at first; test under your ordinary workload. Record RAM and GPU memory use while loading the model and performing a representative task.
- Try a longer real-world prompt with dummy data. A short “hello” does not reveal how a large codebase or long conversation behaves.
- Check latency and heat. Note time to first response, tokens per second if the runtime exposes it, fan noise, and whether the laptop throttles during a longer session.
- Repeat with fewer background apps. This helps determine whether the bottleneck is the model itself or memory pressure from the rest of the workflow.
- Check upgradeability before buying. Many thin laptops have soldered RAM. You may not be able to add memory later.
What a Copilot+ PC badge does—and does not—tell you
Microsoft defines Copilot+ PCs around specific hardware capabilities, including an NPU rated at 40+ TOPS and a minimum memory configuration for that device class. That is a requirement for a category of Windows features and APIs. It does not promise that every open-weight language model will run at a preferred context length or speed, nor that a third-party coding tool will use the NPU.
Microsoft’s Windows ML and Foundry guidance describes several hardware paths—CPU, GPU, and NPU—and notes that requirements vary by model and execution provider. Ask which path your intended app uses. If it falls back to cloud inference, the machine’s RAM may be irrelevant to the model response, and the privacy/data route changes. See our explainer on why local AI is not automatically offline.
Signs you are running out of headroom
- The model refuses to load or reports insufficient memory.
- The runtime silently uses a shorter context than you expected.
- Typing and window switching become sluggish while inference runs.
- The laptop swaps heavily to disk or closes background apps.
- Only a small portion of the model is offloaded to the GPU, causing slow responses.
- A longer prompt makes performance collapse even though a short test seemed fine.
Before assuming the laptop is defective, confirm the selected model, quantization, context setting, and runtime logs. Lowering context can reduce memory use, but it can also make the model forget earlier code or documents. A cloud model may be an option for a non-sensitive task, but understand the data flow and cost before switching.
Bottom line
Buy for the workload, not an “AI PC” sticker. Check model weights, context, runtime buffers, VRAM or unified memory, and the apps you keep open. The smallest model that technically launches may not be pleasant to use; the largest model advertised by a tool may not fit your device. Test the exact model and context, leave headroom for normal work, and verify upgradeability before spending.
Sources
- Ollama: Local coding models and context guidance
- Microsoft Learn: Copilot+ PC and NPU developer guidance
- Microsoft Learn: Windows ML and CPU/GPU/NPU execution
- Microsoft Learn: Compare Windows AI options
Requirements can change as models and runtimes are updated. Check the exact version and configuration you intend to use before purchase.