Open Source
llama.cpp b10566 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Ensures llama.cpp remains accessible across diverse hardware setups.
- →Expands compatibility with Vulkan and OpenVINO, enhancing versatility.
- →Reinforces llama.cpp's position as a flexible inference runtime.
The b10566 release of llama.cpp has been announced, focusing on expanding platform support rather than introducing new features. This update includes compatibility with various architectures on macOS, Linux, Windows, and openEuler, although some configurations like macOS Apple Silicon with KleidiAI and Ubuntu x64 with ROCm 7.14 are disabled. The release emphasizes maintaining accessibility across different systems, with support for Vulkan and OpenVINO on several platforms. This update reinforces llama.cpp's role as a flexible inference runtime for a wide array of hardware environments.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b10541 Release Adds New Features
The b10541 release of llama.cpp enhances developer flexibility with the introduction of the --mmproj-device argument, allowing for more nuanced control over device backends. This update also maintains compatibility with existing setups through the MTMD_BACKEND_DEVICE environment variable and introduces a convenient -mmdev shortflag. These improvements make it easier for developers to manage and load device backends efficiently. While there are no new model architectures in this release, the focus on refining usability ensures that developers can deploy their applications smoothly across different environments.
llama.cpp Releases
Models & Labsmodels
llama.cpp b10545 Release Fixes Tensor API Bug
The latest release of llama.cpp, version b10545, addresses a critical bug in the Tensor API's mat-mat kernel. Previously, the kernel could read out-of-bounds elements when the K dimension wasn't a multiple of 32, leading to potential data corruption or NaN results. This update introduces a dynamic extent for K, ensuring that only valid data is processed, thus enhancing the reliability of matrix operations. This fix is crucial for developers relying on precise tensor computations, especially in environments where K-aligned inputs are not guaranteed.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10568 Release Expands Platform Support
The latest b10568 release of llama.cpp continues its trend of broadening platform compatibility, now incorporating the ggml_rope_set_offset() function. This update partially applies to deepseek2, enhancing its functionality. The release maintains support across a wide array of systems, including macOS, Linux, Windows, and openEuler, with specific configurations for Vulkan, ROCm, and CUDA environments. While no groundbreaking features are introduced, this update solidifies llama.cpp's position as a versatile tool for developers working across diverse hardware setups.
llama.cpp Releases
More in Open Source
© Lev SelectorOpen Sourcemodels
DeepSeek Harness Gains 200,000 GitHub Stars in a Week
DeepSeek Harness has rapidly gained popularity, reaching nearly 200,000 stars on GitHub within a week of its release.
Lev Selector
© GitHub ChangelogOpen Sourcecoding
GitHub Enhances License Data Quality
GitHub has significantly improved the accuracy of license data for software components by integrating package registries like npmjs.org and PyPI into its dependency graph. This shift reduces the reliance on the ClearlyDefined service, which often produced complex and confusing results. By prioritizing registry data, GitHub has halved the number of missing licenses, enhancing the reliability of dependency insights and software bills of materials. This update also simplifies license tracking by using version ranges, making it easier to manage license changes over time.
GitHub Changelog
© NVIDIA BlogOpen Sourceagents
NVIDIA Boosts Local AI with New Open Models
NVIDIA is making strides in the open-source AI ecosystem by releasing several new models and tools that enhance local AI capabilities. Notably, the Cosmos 3 Edge model for robotics and autonomous vehicles, and the MiniMax-H3 model for video and audio generation, are optimized for NVIDIA GPUs, allowing developers to run complex AI tasks locally. This push towards local AI is further supported by the launch of Unsloth Desktop, a comprehensive open-source app for AI model training and inference. These developments signify a shift towards more accessible and efficient AI processing on personal devices, reducing reliance on cloud-based solutions.
NVIDIA Blog