LLMScaleDeepSeek V3: how much VRAM does it need?
DeepSeek V3 has 684.53B parameters in 61 layers. It uses multi-head latent attention (MLA): each layer caches one compressed 576-dimensional latent per token instead of keys and values for every head, which makes its KV cache 24.9× smaller than the usual formula would give. It is a Mixture-of-Experts model with about 37.00B of its 684.53B parameters active per token: memory has to hold all of them, while each token passes through only the active share. It takes up to 163,840 tokens of context; one user at that length needs 10.72 GiB of KV cache at FP16 on top of the weights.
1275.0 GiB of weights, 549 MiB of KV cache and 3.49 GiB of vLLM's overhead. At INT4 the same workload needs 345.2 GiB.
- 61 layers
- hidden 7168
- 128/128 heads
- head_dim 56
- 160k max context
- MoE
- MLA
DeepSeek V3 memory by precision and context
| Precision | 8k context | 32k context | 128k context | 160k context |
|---|---|---|---|---|
| BF16 | 1279.1 GiBKV cache 549 MiB | 1280.7 GiBKV cache 2.14 GiB | 1287.1 GiBKV cache 8.58 GiB | 1289.3 GiBKV cache 10.72 GiB |
| FP8 | 643.3 GiBKV cache 549 MiB | 644.9 GiBKV cache 2.14 GiB | 651.3 GiBKV cache 8.58 GiB | 653.5 GiBKV cache 10.72 GiB |
| INT4 | 345.2 GiBKV cache 549 MiB | 346.9 GiBKV cache 2.14 GiB | 353.3 GiBKV cache 8.58 GiB | 355.4 GiBKV cache 10.72 GiB |
One user, FP16 KV cache and vLLM's overhead on one GPU at its defaults for a card under 70 GB. Quantizing shrinks the weights, not the KV cache, so the saving narrows as the context grows.
Which single GPUs hold it
- BF16 (1279.1 GiB): no single GPU holds it — the fewest that serve it, as the calculator lays them out: 20× H100 SXM 80GB, 12× H200 141GB, 9× B200 180GB.
- FP8 (643.3 GiB): no single GPU holds it — the fewest that serve it, as the calculator lays them out: 10× H100 SXM 80GB, 6× H200 141GB, 5× B200 180GB.
- INT4 (345.2 GiB): Apple M3 Ultra (512GB)
At 8k context for one user, the smallest cards first. A discrete card is budgeted at vLLM's default 92% of its memory, a unified-memory device at the share its GPU can address, and each carries vLLM's overhead at its own defaults.
How many users fit
No single card here holds it at any precision; serving it means splitting it across several GPUs with tensor parallelism.
Fine-tuning DeepSeek V3
| Method | Memory | Smallest GPU |
|---|---|---|
| Full fine-tune | 10203.0 GiB | several GPUs, sharded |
| LoRA (rank 16, attention) | 1278.4 GiB | several GPUs, sharded |
| QLoRA (rank 16, attention) | 332.4 GiB | Apple M3 Ultra (512GB) |
Sequence length 2,048, batch size 1, gradient checkpointing on, Adam. Full fine-tuning keeps 16 bytes per parameter before activations; QLoRA stores the frozen base in 4-bit NF4.
Where the numbers come from
Weights are parameters × bytes per parameter. With MLA the KV cache is layers × (kv_lora_rank + qk_rope_head_dim) × bytes × context × users. For DeepSeek V3 that is 68.6 KiB per token at FP16. These are capacity-planning figures: the serving engine — vLLM, TGI, llama.cpp — and its paged-attention efficiency still move the real number.
More on KV cache size and on GQA, MQA and MLA.