Models & Labs
Granite Speech Plus Support Added in b9768 Release
llama.cpp Releaseshigh confidence
Why it matters
- →Enhances audio processing capabilities with multi-layer concatenation.
- →Fixes and aligns naming conventions for better consistency.
- →Strengthens the framework without introducing new models.
The latest b9768 release of llama.cpp adds support for Granite Speech Plus, focusing on audio processing enhancements. This update includes features like multi-layer concatenation and fixes to naming conventions for feature layers. The release aims to improve the robustness of audio applications without introducing new models. Developers can expect a more aligned and consistent framework for audio processing tasks.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9767 Release Enhances MTP Inference
The b9767 release of llama.cpp introduces significant improvements to MTP inference by optimizing the mat-vec path for small batches, which enhances decoding efficiency. A new barrier in the NUM_COLS loop of the mul-mat-vec process is expected to boost performance. While no new model architectures are included, this update refines the platform's capabilities across macOS, Linux, and Windows. Notably, it supports macOS Apple Silicon, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13. This release continues llama.cpp's focus on performance optimization and compatibility, making it a more powerful tool for developers.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9771 Release Trims Shader Variants
The b9771 release of llama.cpp brings a notable optimization by setting 'mul_mm ALIGNED' as a spec constant, effectively reducing the shader variant explosion and cutting down the binary size. This change is particularly advantageous for developers using Vulkan, as it simplifies the compilation process. While the update doesn't introduce new features, it continues to enhance the platform's compatibility across macOS, Linux, Windows, and openEuler. This release is a step forward in making llama.cpp more efficient and accessible for developers working with different hardware setups, including Apple Silicon, ROCm, and CUDA environments.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9773 Release Expands Platform Support
The b9773 release of llama.cpp continues its trend of broadening platform compatibility, though without major new features. Notably, it includes support for ROCm 7.2 on Ubuntu x64, which is significant for AMD GPU users seeking alternatives to NVIDIA's CUDA. The release also maintains a wide array of builds across macOS, Linux, Windows, and openEuler, ensuring that developers can deploy llama.cpp in many different computing environments. While the update doesn't introduce groundbreaking changes, it solidifies llama.cpp's position as a versatile tool for AI inference across multiple systems.
llama.cpp Releases
More in Models & Labs
© NVIDIA BlogModels & Labsagents
NVIDIA Launches Agent Toolkit for Specialized AI
NVIDIA's new Agent Toolkit is a significant step towards creating specialized AI agents that can be customized and trusted by enterprises. By providing a modular foundation of models, tools, and secure runtime, the toolkit allows businesses to build AI systems tailored to their specific workflows. This development is particularly impactful in industries like life sciences and healthcare, where AI agents can drastically reduce the time needed for complex tasks such as protein design and clinical documentation. The toolkit's open nature ensures that companies can integrate these agents into existing systems, enhancing efficiency and control.
NVIDIA Blog
© The Rundown AIModels & Labsmodels
Sakana AI launches Fugu for model orchestration
Sakana AI's Fugu model introduces a novel approach to AI usage by coordinating multiple models through a single API, addressing challenges like those posed by export controls on Anthropic's models. Fugu is available in two versions: a faster model for everyday tasks and a more robust version for complex applications such as patent research. While Sakana asserts that Fugu performs comparably to leading models, initial feedback suggests it may not yet achieve those standards. This launch represents a shift towards model orchestration, though questions about cost and transparency remain unresolved.
The Rundown AI
© Hugging Face BlogModels & Labscoding
Cross-Origin Storage API in Transformers.js
The proposed Cross-Origin Storage API could revolutionize how web apps handle large files across different origins by using cryptographic hashes instead of URLs for identification. This approach aims to eliminate redundant downloads and storage, which is currently a challenge due to browser cache isolation by origin. By allowing shared resources like AI models and Wasm files to be recognized across different apps, this API could significantly reduce bandwidth and storage usage. Although still in early stages and not natively supported by browsers, developers can experiment with it using a polyfill extension.
Hugging Face Blog