Models & Labs
llama.cpp update: server accepts null sampling params
llama.cpp Releaseshigh confidence
Why it matters
- →Aligns llama.cpp with OpenAI's specification for handling null values.
- →Enhances flexibility for developers by allowing server defaults through null values.
- →Simplifies client-server interactions by treating null values as absent.
Llama.cpp has released an update allowing its server to accept null values for sampling parameters such as temperature and top_p. This change aligns with the OpenAI specification, enabling clients to send null values to use server defaults. The update includes schema validation extensions and a new has_field() function to handle null values effectively. This improvement enhances flexibility and compatibility for developers using llama.cpp.
Read originalMore from llama.cpp Releases
Models & Labsmodels
Llama.cpp b9946 Release Enhances Hexagon Optimizations
The latest b9946 release of llama.cpp focuses on optimizing Hexagon operations, particularly unary operations, to improve performance and efficiency. By introducing tiling for wide rows and replacing divisions with fastdiv, the update aims to prevent VTCM overflow and streamline code execution. The release also includes tracing instrumentation and specialized thread functions to enhance code generation. While no new models are introduced, these technical improvements make llama.cpp more robust and efficient for developers working with Hexagon architectures.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9947 Release Expands Platform Support
The latest b9947 release of llama.cpp continues its trend of broadening platform compatibility, though without major new features. Notably, the release includes support for ROCm 7.2 on Ubuntu x64, which is significant for AMD GPU users seeking alternatives to NVIDIA's CUDA. While KleidiAI support for Apple Silicon remains disabled, the release still covers a wide array of systems, from Windows CUDA 13 to Ubuntu Vulkan. This update solidifies llama.cpp's role as a versatile inference runtime, though it doesn't introduce groundbreaking changes.
