Step 3 · Build plans
Grouped by the tiers from step 2. Lines marked “rule of thumb” are our recommendations, not vendor specs.
Runs: 32B 4-bit with long context; 70B only with CPU offload (slow)
Runs: 70B-class 4-bit for one user; no CUDA, so most training tools do not apply
Runs: Vendor: inference up to 200B, fine-tuning up to 70B on one unit; slower generation than a discrete GPU
Runs: Quantised MoE models of several hundred B in one quiet box; limited concurrency
Runs: 192–384GB: 70B BF16 / FP8 serving for a small team, QLoRA on 70B+
Runs: 70B BF16 at high concurrency; a 600B-class MoE in FP8 on one node (formula estimate)
Runs: More memory per dollar; benchmark first – software maturity differs from CUDA
Runs: 1T-parameter MoE in FP8 on one node – full plan on the next page
Runs: 600B-class MoE in FP8, or ~1T in FP4 (formula estimate)
Runs: Same 288GB per GPU as B300; liquid or air depending on the chassis
Runs: Pre-training and large-scale RL
Runs: Largest reasoning models with very long context; needs liquid-cooling infrastructure
Runs: Previous-generation rack scale; same facility class as GB300
| Hardware | Tier | Memory | Bandwidth | Power | Price | Good for | Sources |
|---|---|---|---|---|---|---|---|
| GeForce RTX 4090NVIDIA | Personal | 24GBGDDR6X | To be verified | 450 W | Street price varies; to be verifiedquote / TBV | 7–32B quantised models on one card; LoRA on 7–8B | nvidia.com |
| GeForce RTX 5090NVIDIA | Personal | 32GBGDDR7 | 1.79 TB/s | 575 W | Launch MSRP $1,999 (Jan 2025); street prices are often higher | 32B-class 4-bit with longer context; a common single-GPU dev box | nvidia.comnvidianews.nvidia.com |
| Mac Studio (M5 Max)Apple | Personal | 128GBunified | 614 GB/s | 480 W (max continuous, whole system) | US Apple Store standard configs from $2,499 (read 2026-10-21) | Quiet and low-power; single-user 70B-class 4-bit inference; no CUDA training | apple.comapple.com |
| DGX Spark (GB10)NVIDIAvendor claim | Personal | 128GBLPDDR5x | 273 GB/s | 240 W PSU (GB10 TDP 140 W) | Founders Edition MSRP raised to $4,699 in Feb 2026; later higher listings reported – to be verifiedquote / TBV | Desktop CUDA dev box: vendor says inference up to 200B, fine-tuning up to 70B; 273 GB/s bandwidth means slower generation than a discrete GPU | nvidia.comforums.developer.nvidia.com |
| Mac Studio (M5 Ultra)Apple | Studio | 512GBunified | 1.2 TB/s | 480 W (max continuous, whole system) | Standard configs up to $6,799; 256/512GB are build-to-order – see Apple | 256/512GB unified memory holds several-hundred-B quantised MoE models; limited concurrent throughput | apple.comapple.com |
| RTX PRO 6000 Blackwell (Workstation)NVIDIA | Studio | 96GBGDDR7 ECC | 1.79 TB/s | 600 W | Quote required (no public price on NVIDIA page)quote / TBV | 96GB per card: 70B FP8 or 100B+ 4-bit; 2–4-card workstations for QLoRA / small full fine-tunes | nvidia.com |
| H100 SXM (HGX H100 ×8)NVIDIA | Team | 80GB × 8HBM3 | 3.35 TB/s | up to 700 W / GPU | Quote required; cloud rental prices on Servicesquote / TBV | 8×80GB = 640GB: 70B BF16 serving, 30–70B full fine-tunes; largest installed base | nvidia.comdocs.nvidia.com |
| H200 SXM (HGX H200 ×8)NVIDIA | Team | 141GB × 8HBM3e | 4.8 TB/s | up to 700 W / GPU | Quote requiredquote / TBV | 8 GPUs ≈ 1.1TB: a 600B+ MoE (DeepSeek-V3 class) in FP8 on one node | nvidia.comdocs.nvidia.com |
| Instinct MI300X (×8 OAM)AMD | Team | 192GB × 8HBM3 | 5.3 TB/s | 750 W / GPU | Quote requiredquote / TBV | 8 GPUs = 1.5TB; ROCm stack; vLLM / SGLang support ROCm – benchmark your own models first | amd.comrocm.docs.amd.com |
| B200 SXM (HGX B200 ×8)NVIDIA | Enterprise | 180GB × 8HBM3e | 8 TB/s | To be verified | Quote requiredquote / TBV | 8 GPUs = 1.44TB; FP4/FP8 inference and mid-to-large training | nvidia.comdocs.nvidia.com |
| B300 SXM (HGX B300 ×8)NVIDIA | Enterprise | 288GB × 8HBM3e | 8 TB/s | 1,100 W / GPU (Dell XE9780 config) | Quote required (no official public price found)quote / TBV | 8 GPUs ≈ 2.3TB: a 1T-parameter MoE in FP8 on one node, long-context reasoning; see the B300 build plan | nvidia.comdocs.nvidia.comdell.com |
| Instinct MI355X (×8 OAM)AMD | Enterprise | 288GB × 8HBM3E | 8 TB/s | 1,400 W TBP / GPU | Quote requiredquote / TBV | 288GB per GPU like B300; e.g. HPE XD685 offers both B300 and MI355X | amd.combuy.hpe.com |
| GB200 NVL72 (rack)NVIDIA | Cluster | 13.4TBHBM3E | 576 TB/s | To be verified | Quote requiredquote / TBV | 72 GPUs + 36 Grace CPUs in one NVLink domain (130 TB/s), liquid-cooled rack | nvidia.com |
| GB300 NVL72 (rack)NVIDIA | Cluster | 20TBHBM3e | 576 TB/s | To be verified | Quote requiredquote / TBV | 72 Blackwell Ultra GPUs + 36 Grace; 20TB GPU memory, 37TB fast memory; fully liquid-cooled | nvidia.com |
Data updated: Compute hardware