Qwen3 Coder 30B A3B: how much VRAM does it need?

Qwen3 Coder 30B A3B has 30.53B parameters in 48 layers. It uses grouped-query attention: 32 query heads share 4 key-value heads, so its KV cache is 8× smaller than full multi-head attention would need. It is a Mixture-of-Experts model with about 3.30B of its 30.53B parameters active per token: memory has to hold all of them, while each token passes through only the active share. It takes up to 262,144 tokens of context; one user at that length needs 24.00 GiB of KV cache at FP16 on top of the weights.

BF16 · 8k context · one user

59.07 GiB

56.87 GiB of weights, 768 MiB of KV cache and 1.44 GiB of vLLM's overhead. At INT4 the same workload needs 18.15 GiB.

48 layers
hidden 2048
32/4 heads
head_dim 128
256k max context
MoE

Qwen3 Coder 30B A3B memory by precision and context

Precision	8k context	32k context	128k context	256k context
BF16	59.07 GiBKV cache 768 MiB	61.32 GiBKV cache 3.00 GiB	70.32 GiBKV cache 12.00 GiB	82.32 GiBKV cache 24.00 GiB
FP8	31.21 GiBKV cache 768 MiB	33.46 GiBKV cache 3.00 GiB	42.46 GiBKV cache 12.00 GiB	54.46 GiBKV cache 24.00 GiB
INT4	18.15 GiBKV cache 768 MiB	20.40 GiBKV cache 3.00 GiB	29.40 GiBKV cache 12.00 GiB	41.40 GiBKV cache 24.00 GiB

One user, FP16 KV cache and vLLM's overhead on one GPU at its defaults for a card under 70 GB. Quantizing shrinks the weights, not the KV cache, so the saving narrows as the context grows.

Which single GPUs hold it

BF16 (59.07 GiB): RTX PRO 5000 Blackwell (72GB), A100 80GB, H100 SXM 80GB, H100 PCIe 80GB
FP8 (31.21 GiB): Apple M4 Pro, A100 40GB, NVIDIA L40S, RTX 6000 Ada
INT4 (18.15 GiB): NVIDIA L4, Apple M4, NVIDIA A30, RTX 3090 / Ti

At 8k context for one user, the smallest cards first. A discrete card is budgeted at vLLM's default 92% of its memory, a unified-memory device at the share its GPU can address, and each carries vLLM's overhead at its own defaults.

How many users fit

GPU	BF16	FP8	INT4
RTX 4090	does not fit	does not fit	5
RTX 5090	does not fit	does not fit	15
RTX 6000 Ada	does not fit	17	35
H100 SXM 80GB	18	55	72
RTX PRO 6000 Blackwell	37	74	92
AMD Ryzen AI Max+ 395 (128GB)	38	76	93
H200 141GB	92	129	147
B200 180GB	138	175	193

Users at 8k context each, on one card. ✗: the weights leave no room for even one.

Fine-tuning Qwen3 Coder 30B A3B

Method	Memory	Smallest GPU
Full fine-tune	456.3 GiB	several GPUs, sharded
LoRA (rank 16, attention)	58.38 GiB	AMD MI210
QLoRA (rank 16, attention)	16.18 GiB	NVIDIA L4

Sequence length 2,048, batch size 1, gradient checkpointing on, Adam. Full fine-tuning keeps 16 bytes per parameter before activations; QLoRA stores the frozen base in 4-bit NF4.

Where the numbers come from

Weights are parameters × bytes per parameter. The KV cache is 2 × layers × key-value heads × head dimension × bytes × context × users. For Qwen3 Coder 30B A3B that is 96.0 KiB per token at FP16. These are capacity-planning figures: the serving engine — vLLM, TGI, llama.cpp — and its paged-attention efficiency still move the real number.

More on KV cache size and on GQA, MQA and MLA.