LLM Size Calculator
Enter the model size in billions of parameters, the precision, the number of layers, the KV width, the context length and the batch size. The calculator estimates the memory for weights, KV cache and overhead.
- GPU memory
- Weights
- KV cache
KV width is KV heads × head size, often 4,096 for 7B models or 1,024 for models with grouped query attention.
Weights take parameters × bytes each. A 7B model at FP16 needs 14 GB for weights. A 4,096 token context with 32 layers and a 4,096 wide KV cache adds 2.1 GB, and 20% overhead brings it to about 19.4 GB, a 24 GB GPU.
LLM memory chart
Weights only. Search for a size.
| Parameters | FP16 (GB) | INT8 (GB) | 4 bit (GB) |
|---|---|---|---|
| 1B | 2 | 1 | 0.5 |
| 3B | 6 | 3 | 1.5 |
| 7B | 14 | 7 | 3.5 |
| 8B | 16 | 8 | 4 |
| 13B | 26 | 13 | 6.5 |
| 34B | 68 | 34 | 17 |
| 70B | 140 | 70 | 35 |
| 180B | 360 | 180 | 90 |
| 405B | 810 | 405 | 202.5 |
GB.
Model size and memory
FP16, 32 layers, 4,096 tokens. Slide the billions of parameters.
The bar shows weights, cache and overhead against the GPU.
How to estimate LLM memory
Model to GPU. Step through it.
Weights
Params × bytes.
Quantising to 4 bit lets big models run on one GPU.
FP16, INT8 or 4 bit
Precision. Compare them.
Full quality.
Half size.
Quarter size.
Lower precision saves memory with a small loss in quality.
Before you deploy
Model checks. Tick each item.
Tick each item as you go.
Long contexts can need more memory than the weights.
What you get
Memory sizing. Each tile explains one result.
GB.
GB.
GB.
Fit.
Rough guide.
Grouped query attention shrinks the KV cache several times.
Frequently Asked Questions
No question matches that search.
How much GPU memory does an LLM need?
About two gigabytes per billion parameters at sixteen bit, plus the KV cache and overhead.
How much memory for a seven billion parameter model?
About fourteen gigabytes for FP sixteen weights, around nineteen with cache and overhead.
What is the KV cache?
Stored keys and values from earlier tokens, so the model does not recompute them.
Does a longer context need more memory?
Yes. The KV cache grows in step with the context length.
Can I run a large model on a small GPU?
With four bit quantisation, many models fit in far less memory.
How much memory does training need?
About sixteen bytes per parameter with the Adam optimiser, before activations.
What is quantisation?
Storing weights with fewer bits to save memory.
Fit the model on your GPU
Estimate the memory before you load a model and choose the precision and context that fit.
More size calculators
Each one has its own calculator, worked examples and answers to common questions.