LLMScalelive Hugging Face data
ENTR

Which LLMs fit on Apple M3 Ultra (512GB)?

Apple M3 Ultra (512GB) shares 512 GB of memory between CPU and GPU; by default the GPU can address about 384 GB of it, and LLMScale budgets 384.0 GiB for a model. Its memory bandwidth, 819 GB/s, is what limits generation speed once a model fits. vLLM has no Metal backend, so models run on it with llama.cpp or MLX. Of the 31 popular open models below, 29 fit at BF16, 29 at FP8 and 31 at INT4 with an 8k context for one user.

Apple · usable memory
384.0 GiB

31 of 31 models fit at INT4, 29 at BF16.

Which LLMs fit on Apple M3 Ultra (512GB)

ModelBF16FP8INT4UsersMax context
Qwen2.5 0.5B Instruct2.63 GiB2.29 GiB2.14 GiB4,068BF1632kBF16
Gemma 3 1B IT4.38 GiB3.73 GiB3.43 GiB1,733BF1632kBF16
Llama 3.2 1B Instruct4.50 GiB3.60 GiB3.17 GiB1,518BF16128kBF16
Llama 3.2 3B Instruct8.91 GiB6.29 GiB5.06 GiB429BF16128kBF16
Phi-3 mini 4k10.33 GiB6.95 GiB5.37 GiB250BF164kBF16
Gemma 3 4B IT18.67 GiB15.29 GiB13.71 GiB334BF16128kBF16
Mistral 7B Instruct v0.317.03 GiB10.53 GiB7.48 GiB367BF1632kBF16
Qwen2.5 Coder 7B18.11 GiB12.03 GiB9.18 GiB837BF1632kBF16
Qwen2.5 7B Instruct18.11 GiB12.03 GiB9.18 GiB837BF1632kBF16
Llama 3.1 8B Instruct18.85 GiB12.35 GiB9.30 GiB366BF16128kBF16
Qwen3 8B19.14 GiB12.67 GiB9.64 GiB325BF1640kBF16
Gemma 2 9B IT23.25 GiB15.50 GiB11.87 GiB138BF168kBF16
Gemma 3 12B IT36.08 GiB25.67 GiB20.79 GiB116BF16128kBF16
Mistral Nemo 12B Instruct27.11 GiB16.95 GiB12.19 GiB286BF16128kBF16
Qwen2.5 14B Instruct32.15 GiB19.84 GiB14.07 GiB235BF1632kBF16
gpt-oss 20B43.44 GiB25.04 GiB17.30 GiB908BF16128kBF16
Gemma 2 27B IT59.82 GiB35.59 GiB24.23 GiB113BF168kBF16
Gemma 3 27B IT66.89 GiB42.65 GiB31.29 GiB73BF16128kBF16
Qwen3 30B A3B60.60 GiB32.74 GiB19.69 GiB432BF1640kBF16
Qwen3 Coder 30B A3B60.60 GiB32.74 GiB19.69 GiB432BF16256kBF16
Qwen2.5 32B Instruct68.00 GiB38.94 GiB25.32 GiB158BF1632kBF16
Qwen3 32B67.75 GiB38.69 GiB25.07 GiB159BF1640kBF16
Mixtral 8x7B Instruct95.13 GiB51.88 GiB31.61 GiB289BF1632kBF16
Llama 3.3 70B Instruct139.5 GiB75.80 GiB45.92 GiB98BF16128kBF16
Llama 3.1 70B Instruct139.6 GiB75.80 GiB45.92 GiB98BF16128kBF16
Qwen2.5 72B Instruct143.8 GiB78.37 GiB47.72 GiB97BF1632kBF16
Llama 4 Scout 17B-16E Instruct209.5 GiB110.2 GiB63.68 GiB117BF16939kBF16
gpt-oss 120B222.4 GiB114.6 GiB65.51 GiB288BF16128kBF16
Mixtral 8x22B Instruct272.2 GiB141.6 GiB80.42 GiB64BF1664kBF16
Llama 3.1 405B Instruct771.2 GiB (does not fit)397.2 GiB (does not fit)221.8 GiB42INT4128kINT4
DeepSeek V31282.8 GiB (does not fit)647.0 GiB (does not fit)349.0 GiB66INT4160kINT4

Memory at 8k context for one user (the model's maximum, where that is shorter), FP16 KV cache and vLLM's overhead at this card's defaults. Users (each at 8k) and maximum context are for the highest precision that fits.

What BF16, FP8 and INT4 mean for memory: quantization and VRAM.

A model that does not fit one card can still run across several: tensor parallelism splits its weights and KV cache between them. The calculator works out how many.

Size a model on Apple M3 Ultra (512GB)

Other GPUs