What model size can my VRAM / RAM run?

Symptoms

• The model fails to load, runs extremely slowly, or the system starts swapping

Common causes

• Weights plus context exceed available VRAM (unified memory on Apple silicon)
• The context is set too long; KV cache grows with length

How to fix

1. Estimate at 4-bit: 7–8B ≈ 5 GB, 14B ≈ 9 GB, 32B ≈ 20 GB, 70B ≈ 40 GB or more (orders of magnitude only; depends on model and quantisation)
2. Leave headroom for context and the OS: 8 GB VRAM suits 7–8B, 16 GB can try 14B, 24 GB can try a 4-bit 32B
3. On a Mac you can use most, but not all, unified memory; MLX (mlx-lm) and llama.cpp both support Apple silicon
4. Set context length to what you need instead of the model’s maximum
5. See parameter counts and licences of open-weight models in our Free models section

Notes

• These are rules of thumb, not our own measurements