GCP

GCP offers genuine single-GPU A100 and H100 VMs — the main reason to pick it over AWS, where those cards only come as whole 8-GPU nodes.

Serving shapes

`gpu_type`	Memory	GPUs per node
`A100-80GB`	80 GB	1
`H100-80GB`	80 GB	1

Creating a deployment

client.deployments.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    source="huggingface",
    name="my-deploy",
    gpu={"gpu_type": "H100-80GB", "gpu_count": 1},
    provider="gcp",
)

Serving runs on on-demand VMs (not the Spot capacity used for GCP training).

Where to go next

All serving providers

Compare shapes, status, and the live CLI catalog.

GPU sizing

Match GPU memory to model size.

AWS

Hot Aisle (AMD MI300X)

Get started

Training

Deployments

Evaluations

CLI

Volumes

Serving shapes

Creating a deployment

Where to go next

All serving providers

GPU sizing

​Serving shapes

​Creating a deployment

​Where to go next

All serving providers

GPU sizing

Serving shapes

Creating a deployment

Where to go next