Models & Labs
Llama.cpp b9158 Release Enhances AMD Support
llama.cpp Releaseshigh confidence
Why it matters
- →Enhances performance for AMD users by optimizing RDNA3 support.
- →Allows efficient operation with larger head sizes, improving versatility.
- →Strengthens llama.cpp's adaptability across different hardware configurations.
The b9158 release of llama.cpp introduces RDNA3 support to the CUDA mma FA kernel, optimizing performance for AMD users. This update allows RDNA3 tensor cores to work efficiently with FP16 accumulation, particularly for head sizes that align with the new tile configurations. Additionally, kernel parameters have been tuned for RDNA3, RDNA4, and CDNA1, enabling better performance for larger head sizes on CDNA. These enhancements make llama.cpp more adaptable and efficient across various hardware setups, especially benefiting those using AMD technology.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9145 release addresses SYCL memory issues
The latest llama.cpp release, b9145, tackles a significant issue with SYCL's memory allocation on multi-GPU systems, particularly those using Intel Arc Pro GPUs. By replacing sycl::malloc_device with zeMemAllocDevice, the update drastically reduces system RAM usage from 60 GiB to just 6.7 GiB for a 15.6 GiB model, preventing out-of-memory crashes without sacrificing performance. This change is crucial for developers working with large models on multi-GPU setups, as it ensures more efficient memory management. The update also includes several improvements and bug fixes, enhancing the robustness of the SYCL backend.
llama.cpp Releases
Models & Labsmodels
Llama.cpp Adds Qwen3.5 Tokenizer Handler
Llama.cpp's latest release enhances its capabilities with a non-backtracking tokenizer handler specifically designed for Qwen3.5. This update significantly improves Unicode tokenization, addressing stack overflow issues that occur with long inputs. By adapting the previous Qwen2 fix to meet Qwen3.5's regex requirements, including support for accent marks, the update ensures more reliable text processing. Developers can now expect more stable performance when handling complex Unicode inputs, benefiting from the robust tokenization across different operating systems and hardware configurations. This means smoother operations on platforms like macOS with KleidiAI, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13.
