Open Source
llama.cpp b10989 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands platform support, making llama.cpp more versatile for developers.
- →Enhances AMD GPU compatibility with ROCm 10.0, reducing reliance on NVIDIA.
- →Focuses on accessibility and utility rather than new model introductions.
The b10989 release of llama.cpp has been announced, featuring expanded support across multiple platforms including macOS, Linux, Windows, and openEuler. Key updates include the addition of CUDA 13.3 libraries for both Ubuntu and Windows, as well as ROCm 10.0 support, which is particularly beneficial for AMD GPU users. Although KleidiAI support is disabled on macOS, the release enhances the tool's versatility for developers working with diverse hardware. This update emphasizes platform compatibility rather than introducing new models.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp b10993 adds ROCm 10 and CUDA 13 builds
This release quietly extends llama.cpp's hardware support to the latest driver stacks, adding official binaries for ROCm 10.0 and CUDA 13 across Linux and Windows. For AMD users, this means native compatibility with newer GPU architectures without manual compilation tweaks, while NVIDIA users gain access to the latest CUDA runtime optimizations. The inclusion of WebGPU in CI signals ongoing work toward browser-based inference, though it remains a background effort for now. There are no new model formats or quantization methods here, just broader infrastructure coverage that keeps llama.cpp relevant as hardware evolves.
llama.cpp Releases
Coding Toolscoding
llama.cpp fixes Metal NaN bug for large activations
A critical precision bug in llama.cpp’s Apple Silicon backend has been patched, resolving total inference failures on models with high-activation ranges like Mistral Small 4. The issue stemmed from f16 saturation during matrix multiplication, which turned entire output tensors into NaN values for inputs exceeding ~32 tokens. By implementing an exact, power-of-two rescaling mechanism in the Metal kernel, the fix restores correctness without significant performance penalties. This ensures local inference on M-series chips remains viable for complex MoE architectures that previously crashed.
