Models & Labs
Llama.cpp b10051 Release Enhances SME Kernel Dispatch
llama.cpp Releaseshigh confidence
Why it matters
- →Ensures accurate kernel dispatch on SME(v1) and SME2 hardware.
- →Improves performance and compatibility across different systems.
- →Addresses a critical issue that could lead to incorrect operations on certain hardware.
Llama.cpp has released version b10051, which introduces a crucial update in kernel dispatch by differentiating between SME and SME2 capabilities. Previously, the system treated SME as a single capability, causing issues on SME(v1)-only hardware due to the use of SME2-specific instructions. The update now includes build-time and runtime distinctions, allowing for accurate kernel dispatch based on the hardware's actual capabilities. This change improves the software's compatibility and performance across various hardware configurations.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b10043 release supports CUDA Virtual Devices
The b10043 release of llama.cpp marks a notable enhancement with the addition of CUDA Virtual Devices, which significantly improves GPU resource management. By removing the NCCL path when virtual devices are in use, the update fine-tunes performance for these specific setups. This release also includes a comprehensive code refactor and the implementation of GPUx2 server CI jobs, reflecting a commitment to better testing and deployment processes. While there are no new model architectures, the update enhances the platform's flexibility across various operating systems, making it more adaptable for developers working with a wide range of hardware configurations.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10045 Release Expands Platform Support
The b10045 release of llama.cpp focuses on broadening its platform compatibility, though it doesn't introduce major new features. This update notably includes Vulkan support for Ubuntu and Windows, alongside ROCm 7.2 for Ubuntu, enhancing GPU utilization options for developers. While KleidiAI support for macOS Apple Silicon remains disabled, the release still covers a wide array of operating systems and architectures, offering developers increased flexibility in deployment. This update solidifies llama.cpp's position as a versatile inference runtime across multiple systems, rather than delivering groundbreaking changes.
