Skip to main content
Hot Aisle serves on AMD MI300X VMs. The headline: 192 GB of HBM3 on a single GPU — enough to serve a 70B model in bf16 on one card, where NVIDIA equivalents need a multi-GPU node.

​
Serving shapes

Serving is single-GPU per deployment for now (multi-GPU ROCm tensor parallel is unvalidated).

​
Creating a deployment

​
Engines

The default vllm engine runs the ROCm vLLM container.
engine="max" is not yet supported on AMD GPUs — a create returns a 400 naming the vllm alternative. (MAX on MI300X currently crash-loops after passing health; the gate lifts once the MAX ROCm path stabilizes.)

​
Notes

  • Boxes run ROCm, not CUDA; the serving stack is the ROCm vLLM docker image, so the OpenAI-compatible surface is identical to NVIDIA deployments.
  • Check live MI300X capacity before creating: veri gpu list --provider hotaisle shows a real-time quantity.
  • Training on the same silicon: see Hot Aisle for training, including MI300X performance tuning.

​
Where to go next

All serving providers

Compare shapes, status, and the live CLI catalog.

Train on MI300X

The training side of the AMD stack.