Calculator · arithmetic shown
How much GPU memory does your model need?
The sum is simple — bytes per parameter × parameters, plus room for the KV cache when serving, or for gradients and optimiser state when training. This page shows every term so you can check it.
Precision
What are you running?
Estimated GPU memory needed
172.0 GB
- Weights: 70B × 2 bytes = 140.0 GB
- Runtime overhead: +20% of weights
- KV cache allowance: 8 × ~0.50 GB = 4.0 GB
These are rules of thumb, not a guarantee: real usage depends on context length, batch size, attention implementation and whether you use LoRA, quantisation or offloading — any of which change the answer substantially.
How many GPUs that takes
| GPU | Memory each | GPUs needed | Prices |
|---|---|---|---|
| GB300 | 279 GB | 1 | 1 providers → |
| B300 | 270 GB | 1 | 3 providers → |
| MI300X | 192 GB | 1 | 3 providers → |
| B200 | 180 GB | 1 | 7 providers → |
| H200 | 141 GB | 2 | 7 providers → |
| GH200 | 96 GB | 2 | 1 providers → |
| RTX PRO 6000 | 96 GB | 2 | 5 providers → |
| H100 | 80 GB | 3 | 15 providers → |
| L40 | 48 GB | 4 | 1 providers → |
| L40S | 48 GB | 4 | 10 providers → |
| A100 | 40 GB | 5 | 10 providers → |
| L4 | 24 GB | 8 | 3 providers → |
| Quadro RTX 6000 | 24 GB | 8 | 1 providers → |
| RTX 4000 Ada | 20 GB | 9 | 1 providers → |
GPU memory figures come from each maker’s spec sheet, linked on the GPU page. Splitting a model across GPUs also costs some memory per GPU, so treat the count as a floor.