Open Source
llama.cpp b9776 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands platform support, increasing accessibility for developers.
- →Enhances AMD GPU compatibility with ROCm 7.2, offering alternatives to CUDA.
- →Reinforces llama.cpp's position as a versatile inference runtime.
The b9776 release of llama.cpp has been announced, featuring expanded support across various platforms. This update includes ROCm 7.2 support for Ubuntu x64, enhancing options for AMD GPU users. The release continues to offer a wide range of builds for macOS, Windows, and Linux, catering to developers' diverse needs. While no new model architectures are introduced, the update reinforces llama.cpp's role as a versatile tool for AI inference across multiple systems.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9767 Release Enhances MTP Inference
The b9767 release of llama.cpp introduces significant improvements to MTP inference by optimizing the mat-vec path for small batches, which enhances decoding efficiency. A new barrier in the NUM_COLS loop of the mul-mat-vec process is expected to boost performance. While no new model architectures are included, this update refines the platform's capabilities across macOS, Linux, and Windows. Notably, it supports macOS Apple Silicon, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13. This release continues llama.cpp's focus on performance optimization and compatibility, making it a more powerful tool for developers.
llama.cpp Releases
Models & Labsmodels
Granite Speech Plus Support Added in b9768 Release
The b9768 release of llama.cpp expands its capabilities by integrating Granite Speech Plus, which enhances audio processing with multi-layer concatenation. This update is particularly relevant for developers focused on audio applications, as it resolves naming inconsistencies and standardizes feature layer usage. While no new models are introduced, the release fortifies the existing framework, making it more reliable for audio tasks. This iteration marks a refinement in the tool's functionality, especially for those utilizing its audio features.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9771 Release Trims Shader Variants
The b9771 release of llama.cpp brings a notable optimization by setting 'mul_mm ALIGNED' as a spec constant, effectively reducing the shader variant explosion and cutting down the binary size. This change is particularly advantageous for developers using Vulkan, as it simplifies the compilation process. While the update doesn't introduce new features, it continues to enhance the platform's compatibility across macOS, Linux, Windows, and openEuler. This release is a step forward in making llama.cpp more efficient and accessible for developers working with different hardware setups, including Apple Silicon, ROCm, and CUDA environments.
llama.cpp Releases
More in Open Source
© Hugging Face BlogOpen Sourcecoding
Hugging Face Automates Weekly Releases with AI
Hugging Face has streamlined its release process for the huggingface_hub Python client, moving from a 4-6 week cycle to weekly releases. This shift is powered by a combination of open-source tools and AI, which drafts release notes and automates mechanical tasks, while humans oversee critical judgment areas. The process is designed to be replicable by other maintainers, emphasizing transparency and adaptability. This change not only accelerates the release cycle but also ensures that updates are consistently delivered without the need for proprietary tools.
Hugging Face Blog
Open Sourcecoding
OpenAI Launches Patch the Planet Initiative
OpenAI's new initiative, Patch the Planet, aims to bolster the security of open-source projects by assisting maintainers in identifying and addressing vulnerabilities. This effort combines AI technology with expert reviews to ensure that open-source software remains robust and secure. By providing tools and support, OpenAI is addressing a critical need in the open-source community, where security can often be overlooked due to resource constraints. This initiative could significantly enhance the reliability of widely-used open-source software, making it safer for developers and users alike.
OpenAI
© The AI Daily BriefOpen Sourcemodels
OpenRouter Launches Fusion for Model Routing
OpenRouter has introduced Fusion, a new tool for model routing in AI systems.
The AI Daily Brief