Qwen3-4B: VRAM & local deployment

Total = weights + 10% + 0.5GB overhead + KV cache for one request (FP16 KV). More concurrent users multiply the KV column.

Bits per weight: BF16 16, FP8 8.5, 4-bit 4.85. “Fits” = total ≤ 90% of memory; Mac GPU can use ~2/3 of unified memory (≤36GB) or ~3/4 (larger). For MoE, all experts must be in memory, so size by total parameters. Method →

Only runtimes the publisher or the runtime’s official library names for this model.

• vLLM — mentioned in the official model card
• SGLang — mentioned in the official model card
• llama.cpp — mentioned in the official model card
• LM Studio — mentioned in the official model card
• MLX — mentioned in the official model card
• Transformers — mentioned in the official model card
• KTransformers — mentioned in the official model card
• Ollama — in the official Ollama library · qwen3:4b

ollama run qwen3:4b

Which inference engine to choose →

9 official repo(s) from Qwen.

• Qwen3-4B main
• Qwen3-4B-Base Base
• Qwen3-4B-GGUF GGUF
• Qwen3-4B-AWQ AWQ
• Qwen3-4B-FP8 FP8
• Qwen3-Embedding-4B-GGUF GGUF
• Qwen3-4B-MLX-4bit MLX
• Qwen3-4B-MLX-8bit MLX
• Qwen3-4B-MLX-bf16 MLX/BF16

• Hugging Face · Qwen/Qwen3-4B
• config.json
• License
• Ollama library · qwen3

Data checked 2025-07-26. Parameters, context and licence are copied from the official repository; memory figures are our estimates.