LLMScalelive Hugging Face data
ENTR

LLM VRAM requirements by model

How much GPU memory 31 popular open models need, computed from each one's own architecture: weights, KV cache and overhead at BF16 and INT4 for one user at 8k context. Each page breaks it down by precision and context, lists the GPUs that hold it and how many users fit, and adds fine-tuning memory.

ModelParametersBF16 · 8kINT4 · 8kMax context
DeepSeek V3684.53B37.00B active1279.1 GiB345.2 GiB160k
Gemma 3 1B IT999.9M3.27 GiB2.31 GiB32k
Gemma 3 4B IT4.30B10.57 GiB5.61 GiB128k
Gemma 2 9B IT9.24B22.48 GiB11.09 GiB8k
Gemma 3 12B IT12.19B26.85 GiB11.55 GiB128k
Gemma 2 27B IT27.20B56.79 GiB21.19 GiB8k
Gemma 3 27B IT27.43B56.53 GiB20.93 GiB128k
Llama 3.2 1B Instruct1.24B3.63 GiB2.29 GiB128k
Llama 3.2 3B Instruct3.21B8.28 GiB4.42 GiB128k
Llama 3.1 8B Instruct8.03B17.71 GiB8.16 GiB128k
Llama 3.3 70B Instruct70.55B137.6 GiB44.01 GiB128k
Llama 3.1 70B Instruct70.55B137.6 GiB44.01 GiB128k
Llama 4 Scout 17B-16E Instruct108.64B17.00B active205.4 GiB59.64 GiB10M
Llama 3.1 405B Instruct405.85B768.3 GiB218.9 GiB128k
Mistral 7B Instruct v0.37.25B15.43 GiB5.88 GiB32k
Mistral Nemo 12B Instruct12.25B26.23 GiB11.31 GiB128k
Mixtral 8x7B Instruct46.70B12.90B active89.46 GiB25.94 GiB32k
Mixtral 8x22B Instruct140.60B39.00B active265.6 GiB73.87 GiB64k
gpt-oss 20B20.91B3.60B active41.22 GiB15.08 GiB128k
gpt-oss 120B116.83B5.10B active220.1 GiB63.24 GiB128k
Phi-3 mini 4k3.82B9.42 GiB4.46 GiB4k
Qwen2.5 0.5B Instruct494.0M1.85 GiB1.36 GiB32k
Qwen2.5 Coder 7B7.62B16.37 GiB7.45 GiB32k
Qwen2.5 7B Instruct7.62B16.37 GiB7.45 GiB32k
Qwen3 8B8.19B18.37 GiB8.87 GiB40k
Qwen2.5 14B Instruct14.77B31.48 GiB13.40 GiB32k
Qwen3 30B A3B30.53B3.30B active59.07 GiB18.15 GiB40k
Qwen3 Coder 30B A3B30.53B3.30B active59.07 GiB18.15 GiB256k
Qwen2.5 32B Instruct32.76B65.64 GiB22.96 GiB32k
Qwen3 32B32.76B65.64 GiB22.96 GiB40k
Qwen2.5 72B Instruct72.70B142.0 GiB45.99 GiB32k

One user, FP16 KV cache and vLLM's overhead on one GPU; models with a shorter maximum context are shown at that maximum.