LLMScaleWhich LLMs fit on AMD Ryzen AI Max+ 395 (128GB)?
AMD Ryzen AI Max+ 395 (128GB) shares 128 GB of memory between CPU and GPU; by default the GPU can address about 96 GB of it, and LLMScale budgets 88.32 GiB for a model, vLLM's default 92% of what the GPU reports. Its memory bandwidth, 256 GB/s, is what limits generation speed once a model fits. Of the 31 popular open models below, 22 fit at BF16, 26 at FP8 and 29 at INT4 with an 8k context for one user.
29 of 31 models fit at INT4, 22 at BF16.
Which LLMs fit on AMD Ryzen AI Max+ 395 (128GB)
| Model | BF16 | FP8 | INT4 | Users | Max context |
|---|---|---|---|---|---|
| Qwen2.5 0.5B Instruct | 2.33 GiB | 2.00 GiB | 1.84 GiB | 918BF16 | 32kBF16 |
| Gemma 3 1B IT | 4.10 GiB | 3.45 GiB | 3.15 GiB | 385BF16 | 32kBF16 |
| Llama 3.2 1B Instruct | 4.00 GiB | 3.10 GiB | 2.67 GiB | 338BF16 | 128kBF16 |
| Llama 3.2 3B Instruct | 8.65 GiB | 6.02 GiB | 4.79 GiB | 92BF16 | 128kBF16 |
| Phi-3 mini 4k | 9.83 GiB | 6.45 GiB | 4.87 GiB | 53BF16 | 4kBF16 |
| Gemma 3 4B IT | 14.67 GiB | 11.29 GiB | 9.71 GiB | 68BF16 | 128kBF16 |
| Mistral 7B Instruct v0.3 | 16.15 GiB | 9.65 GiB | 6.60 GiB | 73BF16 | 32kBF16 |
| Qwen2.5 Coder 7B | 16.95 GiB | 10.87 GiB | 8.02 GiB | 164BF16 | 32kBF16 |
| Qwen2.5 7B Instruct | 16.95 GiB | 10.87 GiB | 8.02 GiB | 164BF16 | 32kBF16 |
| Llama 3.1 8B Instruct | 18.07 GiB | 11.57 GiB | 8.53 GiB | 71BF16 | 128kBF16 |
| Qwen3 8B | 18.80 GiB | 12.33 GiB | 9.30 GiB | 62BF16 | 40kBF16 |
| Gemma 2 9B IT | 23.21 GiB | 15.46 GiB | 11.83 GiB | 25BF16 | 8kBF16 |
| Gemma 3 12B IT | 31.77 GiB | 21.36 GiB | 16.48 GiB | 19BF16 | 128kBF16 |
| Mistral Nemo 12B Instruct | 26.61 GiB | 16.45 GiB | 11.69 GiB | 50BF16 | 128kBF16 |
| Qwen2.5 14B Instruct | 31.91 GiB | 19.60 GiB | 13.84 GiB | 38BF16 | 32kBF16 |
| gpt-oss 20B | 42.41 GiB | 24.01 GiB | 17.62 GiB | 123BF16 | 128kBF16 |
| Gemma 2 27B IT | 57.57 GiB | 33.34 GiB | 21.98 GiB | 11BF16 | 8kBF16 |
| Gemma 3 27B IT | 62.20 GiB | 37.97 GiB | 26.61 GiB | 6BF16 | 128kBF16 |
| Qwen3 30B A3B | 59.90 GiB | 32.04 GiB | 18.99 GiB | 38BF16 | 40kBF16 |
| Qwen3 Coder 30B A3B | 59.90 GiB | 32.04 GiB | 18.99 GiB | 38BF16 | 256kBF16 |
| Qwen2.5 32B Instruct | 66.31 GiB | 37.25 GiB | 23.63 GiB | 12BF16 | 32kBF16 |
| Qwen3 32B | 66.19 GiB | 37.13 GiB | 23.50 GiB | 12BF16 | 40kBF16 |
| Mixtral 8x7B Instruct | 91.95 GiB (does not fit) | 48.70 GiB | 28.42 GiB | 40FP8 | 32kFP8 |
| Llama 3.3 70B Instruct | 138.0 GiB (does not fit) | 74.26 GiB | 44.38 GiB | 6FP8 | 53kFP8 |
| Llama 3.1 70B Instruct | 138.0 GiB (does not fit) | 74.26 GiB | 44.38 GiB | 6FP8 | 53kFP8 |
| Qwen2.5 72B Instruct | 142.5 GiB (does not fit) | 77.08 GiB | 46.43 GiB | 5FP8 | 32kFP8 |
| Llama 4 Scout 17B-16E Instruct | 207.5 GiB (does not fit) | 108.2 GiB (does not fit) | 61.69 GiB | 18INT4 | 150kINT4 |
| gpt-oss 120B | 221.3 GiB (does not fit) | 113.6 GiB (does not fit) | 72.34 GiB | 29INT4 | 128kINT4 |
| Mixtral 8x22B Instruct | 268.5 GiB (does not fit) | 137.9 GiB (does not fit) | 76.73 GiB | 7INT4 | 61kINT4 |
| Llama 3.1 405B Instruct | 768.6 GiB (does not fit) | 394.6 GiB (does not fit) | 219.2 GiB (does not fit) | — | — |
| DeepSeek V3 | 1280.7 GiB (does not fit) | 644.9 GiB (does not fit) | 346.8 GiB (does not fit) | — | — |
Memory at 8k context for one user (the model's maximum, where that is shorter), FP16 KV cache and vLLM's overhead at this card's defaults. Users (each at 8k) and maximum context are for the highest precision that fits.
What BF16, FP8 and INT4 mean for memory: quantization and VRAM.
A model that does not fit one card can still run across several: tensor parallelism splits its weights and KV cache between them. The calculator works out how many.