LLMScaleWhich LLMs fit on which GPU
Which of 31 popular open LLMs fit on 18 GPUs people run them on — consumer and workstation cards, data-center GPUs, Apple Silicon and unified-memory AI PCs. Each page lists every model with its memory at BF16, FP8 and INT4, how many users fit and the longest context it can hold.
| GPU | Type | Usable | Models at BF16 | Models at INT4 |
|---|---|---|---|---|
| RTX 3060 | Consumer | 11.04 GiB | 6 | 11 |
| RTX 4060 Ti 16GB | Consumer | 14.72 GiB | 6 | 15 |
| RTX 3090 / Ti | Consumer | 21.71 GiB | 11 | 20 |
| RTX 4090 | Consumer | 21.76 GiB | 11 | 20 |
| AMD RX 7900 XTX | Consumer | 22.08 GiB | 11 | 20 |
| RTX 5090 | Consumer | 28.87 GiB | 14 | 23 |
| NVIDIA L40S | Workstation | 40.81 GiB | 15 | 23 |
| RTX 6000 Ada | Workstation | 43.69 GiB | 16 | 23 |
| RTX A6000 | Workstation | 43.73 GiB | 16 | 23 |
| A100 80GB | Data center | 72.82 GiB | 22 | 28 |
| H100 SXM 80GB | Data center | 72.82 GiB | 22 | 28 |
| RTX PRO 6000 Blackwell | Data center | 87.37 GiB | 22 | 29 |
| AMD Ryzen AI Max+ 395 (128GB) | AI PC (unified) | 88.32 GiB | 22 | 29 |
| Apple M4 Max (128GB) | Apple | 96.00 GiB | 23 | 29 |
| NVIDIA DGX Spark (GB10) | AI PC (unified) | 111.9 GiB | 23 | 29 |
| H200 141GB | Data center | 128.6 GiB | 23 | 29 |
| B200 180GB | Data center | 164.1 GiB | 26 | 29 |
| Apple M3 Ultra (512GB) | Apple | 384.0 GiB | 29 | 31 |
Out of 31 models, at 8k context for one user. Discrete cards are budgeted at vLLM's default 92% of their memory, unified-memory devices at the share their GPU can address, each with vLLM's overhead at its own defaults.