Compare GPU specifications side by side
Put two to four GPUs side by side, as their datasheets have them: architecture, memory and bandwidth, tensor throughput, power and how they connect. Each figure comes from the vendor where the vendor publishes it.
RTX 5090 Consumer · NVIDIA | H100 SXM 80GB Data center · NVIDIA | NVIDIA DGX Spark (GB10) AI PC (unified) · NVIDIA | |
|---|---|---|---|
| Chip | |||
| ArchitectureⓘThe architecture, die and manufacturing node. The architecture decides which number formats the tensor cores run natively — FP8 from Hopper, Ada and CDNA 3, FP4 from Blackwell and CDNA 4. | Blackwell · GB202 · TSMC 4N | Hopper · GH100 · TSMC 4N | Blackwell · GB10 · TSMC 3 nm |
| CoresⓘShader cores as the vendor counts them. Vendors count differently, so this compares cards of one vendor and generation, not across them. | 21,760 CUDA cores | 16,896 CUDA cores | 6,144 CUDA cores |
| LaunchedⓘWhen the vendor launched or began shipping it. | Jan 2025 | Mar 2022 | Oct 2025 |
| Memory | |||
| MemoryⓘOn-board memory as the vendor lists it. A unified-memory device shares its pool between CPU and GPU, and by default its GPU can address only part of it. | 32 GB | 80 GB | 128 GB unified120 GB of it to the GPU by default |
| Memory typeⓘThe memory technology and the width of its interface. Bandwidth is the two multiplied with the memory clock: HBM is wide and slow, GDDR narrow and fast, LPDDR is shared with the CPU. | GDDR7 · 512-bit | HBM3 · 5,120-bit | LPDDR5X-8533 · 256-bit |
| Memory bandwidthⓘHow fast the GPU reads its own memory. Generating a token reads every weight once, so for an LLM this sets the speed one user sees. | 1,792 GB/s | 3,350 GB/s | 273 GB/s |
| Tensor throughputdense, without sparsity | |||
| BF16 tensorⓘDense 16-bit tensor throughput from the vendor's datasheet. For an LLM it sets how fast prompts are read and how far a large batch scales. | 210 TFLOPS | 990 TFLOPS | 125 TFLOPS |
| FP8 tensorⓘDense FP8 tensor throughput. A card without FP8 tensor cores still saves memory with FP8 weights but computes them at 16-bit. | 419 TFLOPS | 1,979 TFLOPS | 250 TFLOPS |
| FP4 tensorⓘDense FP4 tensor throughput (Blackwell, CDNA 4). The engine counts 4-bit weights as computing in 16-bit, the conservative reading. | 1,676 TFLOPS | none | 500 TFLOPS |
| Platform | |||
| PowerⓘThe vendor's maximum board, module or package power. A superchip or system-on-chip's figure includes its CPU, and Apple publishes only the whole Mac's maximum at the wall, so these are not like for like with a graphics card. | 575 W | 700 W | 140 W · CPU included |
| Form factorⓘHow the device is built and plugs in: a PCIe card, an SXM or OAM module on a baseboard, or a system-on-chip. | PCIe 5.0 x16 | SXM5 | SoC |
| GPU-to-GPU linkⓘHow two of these talk to each other, both directions together as vendors quote it. Tensor parallelism exchanges activations every layer, so a fast link matters once a model spans cards; over PCIe alone it scales less well. Thunderbolt, USB4 and Ethernet are quoted in gigabits. | none — PCIe only | NVLink 4 · 900 GB/s | ConnectX-7 · 200 Gb/s |
| MIG partitionsⓘMulti-Instance GPU: how many isolated slices of memory and compute the card can be split into, each able to serve its own model. | none | up to 7 | none |
| Serves withⓘWhat serves models on it. vLLM runs on CUDA and ROCm; Apple Silicon has no vLLM backend, so llama.cpp or MLX. | vLLM · CUDA | vLLM · CUDA | vLLM · CUDA |
Vendor figures. Tensor throughput is dense, halved where only the sparse figure is printed; a value no source settles is shown as —. A unified-memory device shares its memory with the CPU. Apple publishes no chip power, so a Mac's figure is the whole computer at the wall. To size a model on any of these cards, use the calculator.
What the comparison shows
Pick two to four of the 43 devices LLMScale knows: consumer and workstation cards, data-center GPUs, Apple Silicon and unified-memory AI PCs. Each column is that device's datasheet: architecture, die and process, cores and launch; memory with its type, bus width and bandwidth; dense tensor throughput at BF16, FP8 and FP4; power, form factor, the link between two of them, MIG partitions and the engine that serves models on it.
Each fact comes from the vendor's product page, datasheet or whitepaper where the vendor publishes it, and from a named secondary source where it does not; a value no source settles is left blank rather than guessed. The best value of the memory, bandwidth and tensor rows is highlighted when the columns differ. To see what a model needs on any of these cards, open the VRAM calculator.
Questions
- Which of these numbers matters most for running an LLM?
- Memory decides what fits: the weights, plus a KV cache for every user's context. Bandwidth decides how fast one user sees tokens, because each token reads every weight once; an RTX 5090 at 1,792 GB/s generates about twice as fast for one user as an L40S at 864 GB/s, although the L40S is rated higher at BF16. Tensor throughput decides how fast prompts are read and how far a large batch scales.
- Why is some TFLOPS figure lower here than on the vendor's page?
- NVIDIA often prints tensor throughput with 2:4 structured sparsity, which doubles the figure but applies only to weights pruned to that pattern — almost no served LLM is. LLMScale lists the dense rate, halving the sparse one where that is all the vendor prints. GeForce cards are listed at the FP32-accumulate rate, half the FP16-accumulate one.
- What does unified memory mean for an LLM?
- The CPU and GPU share one pool, so a small device can hold a large model — but the GPU addresses only part of it by default: three-quarters on a Mac from 36 GB up and two-thirds below, 96 of 128 GB on Strix Halo under Windows, nearly all of it on DGX Spark. Their bandwidth is also a fraction of a graphics card's, so they hold big models and run them slowly.