Models & Labs

vLLM v0.30.1rc0 adds AMD MI355 NVFP4 support

vLLM ReleasesSeptember 24, 2026high confidence

Why it matters

→Enables NVFP4 quantization on AMD MI355 GPUs for high-throughput inference.
→Reduces the performance gap between AMD and NVIDIA in the vLLM ecosystem.
→Allows enterprises to utilize newer AMD hardware for cost-effective model serving.

vLLM has released version 0.30.1rc0, introducing support for dense NVFP4 and MoRI kernel mirrors on AMD's MI355 GPU architecture. The update, signed off by AMD engineer Andreas Karatzas, ensures that the latest high-performance inference kernels are available for this hardware generation. This release candidate aims to maintain feature parity with NVIDIA counterparts, allowing developers to leverage advanced quantization techniques on AMD systems. The change is part of an ongoing effort to broaden vLLM's hardware compatibility beyond CUDA-centric environments.

Read original

More from vLLM Releases

Models & Labsmodels

vLLM v0.30.0: DeepSeek V4.1 and Fast Start

This release is a massive infrastructure overhaul for serving the latest reasoning models. The headline feature is native support for DeepSeek-V4.1-Flash, storing its entire KV cache in MXFP8 on SM100 hardware to drastically reduce memory overhead. For operators tired of slow cold starts, the new Fast Start daemon caches post-quantized weights in GPU memory, allowing engines to map over CUDA IPC instead of reloading from disk. It also brings HiSparse, a host-resident tier that spills KV pages to pinned host memory under pressure, effectively expanding usable context windows without buying more GPUs.

vLLM ReleasesSep 22, 2026

More in Models & Labs

Models & Labsother

llama.cpp b11149 adds backend testing and ROCm 10 support

This release quietly solidifies llama.cpp’s position as the universal inference runtime by adding explicit ROCm 10.0 builds for both Linux and Windows. The inclusion of CUDA 13.4 alongside the existing 12.x variants ensures compatibility with the latest NVIDIA driver stacks without forcing users to stick to older libraries. More importantly, the new backend testing infrastructure means these diverse hardware configurations are now validated systematically rather than left to chance. This reduces fragmentation for developers running on AMD or newer NVIDIA cards who previously had to troubleshoot build issues manually.

llama.cpp ReleasesSep 24, 2026

Models & Labsother

llama.cpp b11154 improves testing and adds CUDA 13

This release refines the internal test suite for better visibility, but the real signal is platform expansion. CUDA 13 builds are now available across Linux and Windows, giving developers early access to the latest NVIDIA stack without waiting for stable drivers. AMD ROCm support also advances with version 10.0 binaries, keeping pace with hardware shifts. While KleidiAI on Apple Silicon is currently disabled, the core inference runtime remains robust across major architectures. This is a maintenance-heavy update that ensures compatibility with cutting-edge GPU libraries.

llama.cpp ReleasesSep 24, 2026

Models & Labsother

llama.cpp b11155 adds ROCm 10 and CUDA 13 support

This release quietly extends llama.cpp's hardware compatibility to the latest NVIDIA and AMD stacks. By adding default builds for CUDA 13.4 and ROCm 10.0, it ensures local inference stays viable on cutting-edge GPUs without manual compilation. The update also stabilizes Apple Silicon builds by disabling KleidiAI, likely due to stability concerns. For builders running newer hardware, this is the essential patch to avoid compatibility errors.

llama.cpp ReleasesSep 24, 2026