LLMScalelive Hugging Face data
ENTR

Gemma 3 1B IT: how much VRAM does it need?

Gemma 3 1B IT has 999.9M parameters in 26 layers. It uses multi-query attention: all 4 query heads share a single key-value head, so its KV cache is 4× smaller than full multi-head attention would need. It takes up to 32,768 tokens of context; one user at that length needs 236 MiB of KV cache at FP16 on top of the weights. Its repository is gated on Hugging Face, so LLMScale ships its architecture in a bundled database.

BF16 · 8k context · one user
3.27 GiB

1.86 GiB of weights, 140 MiB of KV cache and 1.27 GiB of vLLM's overhead. At INT4 the same workload needs 2.31 GiB.

  • 26 layers
  • hidden 1152
  • 4/1 heads
  • head_dim 256
  • 32k max context

Gemma 3 1B IT memory by precision and context

Precision8k context32k context
BF163.27 GiBKV cache 140 MiB3.36 GiBKV cache 236 MiB
FP82.62 GiBKV cache 140 MiB2.71 GiBKV cache 236 MiB
INT42.31 GiBKV cache 140 MiB2.41 GiBKV cache 236 MiB

One user, FP16 KV cache and vLLM's overhead on one GPU at its defaults for a card under 70 GB. Quantizing shrinks the weights, not the KV cache, so the saving narrows as the context grows.

Which single GPUs hold it

At 8k context for one user, the smallest cards first. A discrete card is budgeted at vLLM's default 92% of its memory, a unified-memory device at the share its GPU can address, and each carries vLLM's overhead at its own defaults.

How many users fit

GPUBF16FP8INT4
RTX 4090135140142
RTX 5090187192194
RTX 6000 Ada295300302
H100 SXM 80GB314317318
RTX PRO 6000 Blackwell381383385
AMD Ryzen AI Max+ 395 (128GB)385388389
H200 141GB569572573
B200 180GB728731733

Users at 8k context each, on one card. ✗: the weights leave no room for even one.

Fine-tuning Gemma 3 1B IT

MethodMemorySmallest GPU
Full fine-tune16.01 GiBNVIDIA L4
LoRA (rank 16, attention)3.02 GiBRTX 3060
QLoRA (rank 16, attention)1.64 GiBRTX 3060

Sequence length 2,048, batch size 1, gradient checkpointing on, Adam. Full fine-tuning keeps 16 bytes per parameter before activations; QLoRA stores the frozen base in 4-bit NF4.

Where the numbers come from

Weights are parameters × bytes per parameter. The KV cache is 2 × layers × key-value heads × head dimension × bytes × context × users. For Gemma 3 1B IT that is 26.0 KiB per token at FP16. 22 of its 26 layers attend over a sliding window of the last 512 tokens and hold no more than that, so past it each further token adds 4.0 KiB, the other 4 layers' share. These are capacity-planning figures: the serving engine — vLLM, TGI, llama.cpp — and its paged-attention efficiency still move the real number.

More on KV cache size and on GQA, MQA and MLA.

Work with Gemma 3 1B IT

Other Gemma models