AMD Instinct MI300X

Manufacturer specification reference

Architecture

CDNA 3AMD Instinct MI300X specifications (external source) · AMD Instinct GPUs: MLPerf Inference v5.1 (external source)

Category

Individual acceleratorAMD Instinct MI300X specifications (external source) · AMD Instinct GPUs: MLPerf Inference v5.1 (external source)

Accelerator memory

192 GBPer OAM accelerator; not the eight-GPU baseboard or host memory.AMD Instinct MI300X specifications (external source)

Memory type

HBM3AMD Instinct MI300X specifications (external source) · AMD Instinct GPUs: MLPerf Inference v5.1 (external source)

Memory bandwidth

5.3 TB/sManufacturer peak theoretical GPU memory bandwidth per accelerator.AMD Instinct MI300X specifications (external source)

Power

750 WManufacturer TBP field labeled Peak; module boundary, not complete server draw.AMD Instinct MI300X specifications (external source)

Precision formats

FP64, FP32, TF32, FP16, BF16, FP8, INT8AMD Instinct MI300X specifications (external source)

Cooling / integration

Passive OAM; host system must provide thermal management.AMD Instinct MI300X specifications (external source) · AMD Instinct GPUs: MLPerf Inference v5.1 (external source)

Configuration boundaries

• Published FLOPS are peak theoretical specifications; they are not model serving benchmarks.
• FP8 format behavior is architecture-specific; a numeric precision label alone is not a deployment compatibility guarantee.

Published arithmetic throughput

Deployment considerations

• Match ROCm, kernel, framework and model versions against the current support matrix.
• Weight-memory estimates exclude KV cache and serving overhead; reserve runtime headroom.
• Check collectives, scale-up topology and model parallelism on the intended OEM system.

Software ecosystem

ROCm. Confirm the exact model, runtime release, server integration and vendor support matrix before evaluating a deployment.