Open Source
llama.cpp b9150 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands platform support, increasing versatility for developers.
- →Enhances performance optimization across diverse hardware setups.
- →Reinforces llama.cpp's role as a flexible inference runtime.
The latest b9150 release of llama.cpp introduces expanded platform support, including macOS Apple Silicon with KleidiAI and various Linux configurations like Ubuntu with ROCm 7.2. Windows users benefit from enhanced support with CUDA 12 and 13 DLLs. This update focuses on broadening compatibility rather than introducing new features, reinforcing llama.cpp's role as a versatile tool for developers. The release allows for optimized performance across a wider array of hardware setups.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9145 release addresses SYCL memory issues
The latest llama.cpp release, b9145, tackles a significant issue with SYCL's memory allocation on multi-GPU systems, particularly those using Intel Arc Pro GPUs. By replacing sycl::malloc_device with zeMemAllocDevice, the update drastically reduces system RAM usage from 60 GiB to just 6.7 GiB for a 15.6 GiB model, preventing out-of-memory crashes without sacrificing performance. This change is crucial for developers working with large models on multi-GPU setups, as it ensures more efficient memory management. The update also includes several improvements and bug fixes, enhancing the robustness of the SYCL backend.
llama.cpp Releases
Models & Labsmodels
Llama.cpp Adds Qwen3.5 Tokenizer Handler
Llama.cpp's latest release enhances its capabilities with a non-backtracking tokenizer handler specifically designed for Qwen3.5. This update significantly improves Unicode tokenization, addressing stack overflow issues that occur with long inputs. By adapting the previous Qwen2 fix to meet Qwen3.5's regex requirements, including support for accent marks, the update ensures more reliable text processing. Developers can now expect more stable performance when handling complex Unicode inputs, benefiting from the robust tokenization across different operating systems and hardware configurations. This means smoother operations on platforms like macOS with KleidiAI, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13.
