Gemma 2 27B IT: how much VRAM does it need?

Gemma 2 27B IT has 27.20B parameters in 46 layers. It uses grouped-query attention: 32 query heads share 16 key-value heads, so its KV cache is 2× smaller than full multi-head attention would need. It takes up to 8,192 tokens of context; one user at that length needs 2.88 GiB of KV cache at FP16 on top of the weights. Its repository is gated on Hugging Face, so LLMScale ships its architecture in a bundled database.

Gemma 2 27B IT memory by precision and context

One user, FP16 KV cache and vLLM's overhead on one GPU at its defaults for a card under 70 GB. Quantizing shrinks the weights, not the KV cache, so the saving narrows as the context grows.

Which single GPUs hold it

• BF16 (56.79 GiB): AMD MI210, RTX PRO 5000 Blackwell (72GB), A100 80GB, H100 SXM 80GB
• FP8 (32.55 GiB): Apple M4 Pro, A100 40GB, NVIDIA L40S, RTX 6000 Ada
• INT4 (21.19 GiB): Apple M4, NVIDIA A30, RTX 3090 / Ti, RTX 4090

At 8k context for one user, the smallest cards first. A discrete card is budgeted at vLLM's default 92% of its memory, a unified-memory device at the share its GPU can address, and each carries vLLM's overhead at its own defaults.

How many users fit

Users at 8k context each, on one card. ✗: the weights leave no room for even one.

Fine-tuning Gemma 2 27B IT

Sequence length 2,048, batch size 1, gradient checkpointing on, Adam. Full fine-tuning keeps 16 bytes per parameter before activations; QLoRA stores the frozen base in 4-bit NF4.

Where the numbers come from

Weights are parameters × bytes per parameter. The KV cache is 2 × layers × key-value heads × head dimension × bytes × context × users. For Gemma 2 27B IT that is 368.0 KiB per token at FP16. 23 of its 46 layers attend over a sliding window of the last 4,096 tokens and hold no more than that, so past it each further token adds 184.0 KiB, the other 23 layers' share. These are capacity-planning figures: the serving engine — vLLM, TGI, llama.cpp — and its paged-attention efficiency still move the real number.

More on KV cache size and on GQA, MQA and MLA.

Work with Gemma 2 27B IT

• Size it for your workload
• Fine-tuning memory
• vLLM parameters
• Model anatomy
• Its config.json, explained
• What its files say about serving it

Other Gemma models

• Gemma 3 1B IT
• Gemma 3 4B IT
• Gemma 2 9B IT
• Gemma 3 12B IT
• Gemma 3 27B IT