Models & Labs
Llama.cpp Adds Granite-Speech Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands llama.cpp's capabilities to include audio processing.
- →Ensures precise token-for-token matching with HF transformers.
- →Enhances versatility for developers working with audio data.
Llama.cpp has released an update adding support for IBM's Granite-Speech, a move that broadens its audio processing capabilities. The update includes a Conformer encoder with Shaw relative position encoding and a QFormer projector, which compresses audio data efficiently. This ensures accurate token-for-token matching with HF transformers on audio clips. The update signifies a notable enhancement in llama.cpp's functionality, making it a more comprehensive tool for developers.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9041 Release Expands Platform Support
The latest b9041 release of llama.cpp continues its trend of broadening platform compatibility, making it a versatile choice for developers across different environments. Notably, this update includes support for macOS Apple Silicon with KleidiAI enabled, as well as expanded Vulkan and ROCm 7.2 support on Ubuntu. This release doesn't introduce new models but focuses on enhancing the runtime's adaptability across various hardware configurations. By doing so, llama.cpp strengthens its position as a go-to inference runtime for developers seeking flexibility beyond NVIDIA's CUDA ecosystem.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9047 release focuses on device memory handling
The b9047 release of llama.cpp enhances how device memory is managed, particularly for GPUs with unknown configurations. By ensuring that memory fit for unknown GPUs is set to zero and maintaining a fallback for non-GPU devices, the update boosts stability and reliability. This release continues to support a broad array of operating systems, including macOS with KleidiAI enabled, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13. While it doesn't introduce groundbreaking features, these refinements make llama.cpp a more dependable tool for developers working across different hardware environments.
More in Models & Labs
© Hugging Face BlogModels & Labsmodels
vLLM V1 Achieves Backend Parity with V0
The transition from vLLM V0 to V1 represents a major backend overhaul, prioritizing parity before modifying reinforcement learning objectives. By resolving issues such as processed rollout logprobs and runtime defaults, the vLLM team ensured that V1's outputs meet the expectations set by V0. This approach demonstrates the critical role of backend accuracy in preserving training integrity. With these adjustments, V1 now mirrors V0's behavior, creating a stable foundation for future enhancements in RL objectives without the complications of backend discrepancies.
Hugging Face Blog
© TechCrunch AI