LLM Opex

Which GPU instance or cluster can actually hold it?

A model fits when its weights, plus the memory its attention cache needs for your context length and the requests you serve at once, are within the GPU memory of an instance or a cluster. Anything too small is left out; what fits is listed per cloud with its price.

Memory needed and the cheapest hardware that fits, by model

At the precision the weights ship in, with the default allowance for the attention cache; list prices, on demand.
ModelParametersPrecisionGPU memory neededCheapest instance that fits
OpenAI gpt-oss 120B116.83 Bint473 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
GLM 5.3753.33 Bfp8941.7 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Kimi K32779.93 Bpublished size1951.1 GBOCI BM.GPU.MI355X.8: 8 × MI355X, $68.80 an hour
GLM 5.2753.33 Bbf161883.3 GBOCI BM.GPU.MI355X.8: 8 × MI355X, $68.80 an hour
Llama 3.3 70B70.55 Bbf16176.4 GBCore42 mi300x.1x: 1 × MI300X, $3.50 an hour
OpenAI gpt-oss 20B20.91 Bint413.1 GBHF Endpoints aws.nvidia-t4.x1: 1 × T4, $0.50 an hour
GLM 5.3 Flash321.32 Bfp8401.6 GBCrusoe a100-sxm.8x: 8 × A100, $18.40 an hour
Llama 4 Maverick 17B401.58 Bbf161003.9 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Qwen3.8 27B27.78 Bbf1669.5 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
MiniMax M3427.04 Bbf161067.6 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
DeepSeek V4 Pro1598.84 Bpublished size1080.9 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
DeepSeek V4 Pro 08131650.5 Bpublished size1115.9 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
DeepSeek V4.1 Flash763.21 Bpublished size637.9 GBCrusoe a100-sxm.8x: 8 × A100, $18.40 an hour
Gemma 4 26B A4B25.81 Bbf1664.5 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
Llama 3.1 8B8.03 Bbf1620.1 GBHF Endpoints gcp.nvidia-l4.x1: 1 × L4, $0.70 an hour
Llama 4 Scout 17B108.64 Bbf16271.6 GBNebius gpu-b300-sxm.1gpu-24vcpu-346gb: 1 × B300, $9.50 an hour
Kimi K2.51026.88 Bpublished size744 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Kimi K2.61026.88 Bpublished size744 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Qwen3 32B32.76 Bbf1681.9 GBNebius gpu-rtx6000-a.1gpu-24vcpu-218gb: 1 × RTX PRO 6000, $1.80 an hour
DeepSeek V3.2685.36 Bfp8856.7 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
DeepSeek V4 Flash290.94 Bpublished size199.5 GBDeepInfra b300.1x: 1 × B300, $5.89 an hour
DeepSeek V4 Flash 0731304.18 Bpublished size208.6 GBDeepInfra b300.1x: 1 × B300, $5.89 an hour
Llama 3.1 70B70.55 Bbf16176.4 GBCore42 mi300x.1x: 1 × MI300X, $3.50 an hour
Mistral Small23.57 Bbf1658.9 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
MiniMax M2.5228.69 Bfp8285.9 GBNebius gpu-b300-sxm.1gpu-24vcpu-346gb: 1 × B300, $9.50 an hour
DeepSeek V3.1684.53 Bfp8855.7 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Gemma 3 27B27.43 Bbf1668.6 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
Llama 3.1 405B405.85 Bbf161014.6 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Phi-414.66 Bbf1636.6 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
Mistral 7B7.25 Bbf1618.1 GBHF Endpoints gcp.nvidia-l4.x1: 1 × L4, $0.70 an hour
Kimi K2.7 Code1026.88 Bpublished size744 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
GLM 5753.86 Bbf161884.7 GBOCI BM.GPU.MI355X.8: 8 × MI355X, $68.80 an hour
Hermes 4 405B405.85 Bbf161014.6 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Qwen3 235B A22B Instruct 2507235.09 Bbf16587.7 GBCrusoe a100-sxm.8x: 8 × A100, $18.40 an hour
Qwen3 30B A3B Instruct 250730.53 Bbf1676.3 GBDeepInfra a100.1x: 1 × A100, $1.49 an hour
Qwen3.5 397B A17B403.4 Bbf161008.5 GBCrusoe mi300x.8x: 8 × MI300X, $27.60 an hour
Mistral Small 4119.4 Bfp8149.2 GBCore42 mi300x.1x: 1 × MI300X, $3.50 an hour
Qwen3 235B A22B235.09 Bbf16587.7 GBCrusoe a100-sxm.8x: 8 × A100, $18.40 an hour
GLM 5.1753.86 Bbf161884.7 GBOCI BM.GPU.MI355X.8: 8 × MI355X, $68.80 an hour

Compare what each cloud charges for a model.