What hardware you need: minimum, recommended and top-end

What you'll see in this lesson

1. Two numbers rule: how much memory and how fast

   Memory decides which model fits; its bandwidth, how many tokens per second it writes. The processor matters much less.

2. Minimum: an 8 or 12 GB graphics card

   With 8 GB you run models of about 4 to 8 billion parameters at 4 bits; with 12 GB, ones of about 12 billion fit.

3. Recommended to start: 16 GB for about €500

   The 16 GB RTX 5060 Ti is the most sensible buy for local AI: more memory than the 5070 and quite a bit cheaper.

4. 24 and 32 GB: used RTX 3090 or RTX 5090

   With 24–32 GB, Qwen3.8-27B fits at 4 bits with long context: the quality sweet spot that fits at home.

5. Top of the line in a PC: RTX PRO 6000 with 96 GB

   NVIDIA's pro card has 96 GB and the bandwidth of a 5090: 120-billion models entirely on the graphics card.

6. 128 GB shared: DGX Spark and Strix Halo

   Mini PCs with 128 GB of unified memory: MoEs of over 100 billion fit, but they read at about 256–273 GB/s.

7. Mac Studio with M5 Max or M5 Ultra: fast memory

   The M5 Max and M5 Ultra chips read at 614 and 1,228 GB/s: the Ultra gets close to an RTX 5090, but with far more possible memory.

8. What to buy based on the model you want to run

   Start with the model you need and choose the memory; the rest of the machine almost decides itself.

9. Summary

   Choose the model first, calculate its memory and buy the machine that has it with the bandwidth you need.