Models & Labs
llama.cpp b9049 Release Supports MiniCPM-V 4.6
llama.cpp Releaseshigh confidence
Why it matters
- →Supports the latest MiniCPM-V 4.6 model, enhancing compatibility.
- →Includes bug fixes and improvements, increasing reliability.
- →Expands platform support, making it more versatile for developers.
The llama.cpp project has released version b9049, which now supports MiniCPM-V 4.6. This update includes several bug fixes and improvements, such as enabling flash attention support and refining code style. The release also broadens platform compatibility, covering macOS, Linux, and Windows, with specific enhancements for Apple Silicon and Vulkan. These changes aim to improve the tool's performance and usability for developers working with AI models.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b9041 Release Expands Platform Support
The latest b9041 release of llama.cpp continues its trend of broadening platform compatibility, making it a versatile choice for developers across different environments. Notably, this update includes support for macOS Apple Silicon with KleidiAI enabled, as well as expanded Vulkan and ROCm 7.2 support on Ubuntu. This release doesn't introduce new models but focuses on enhancing the runtime's adaptability across various hardware configurations. By doing so, llama.cpp strengthens its position as a go-to inference runtime for developers seeking flexibility beyond NVIDIA's CUDA ecosystem.
llama.cpp Releases
Models & Labsmodels
Llama.cpp Adds Granite-Speech Support
Llama.cpp's latest update expands its functionality by integrating IBM's Granite-Speech, significantly enhancing its audio processing capabilities. The update features a Conformer encoder with Shaw relative position encoding and a QFormer projector, which efficiently compresses audio data into the LLM embedding space. This ensures precise token-for-token matching with HF transformers on audio clips, demonstrating its robustness. By incorporating these advanced audio processing techniques, llama.cpp becomes a more versatile tool for developers, extending its utility beyond text to include sophisticated audio data handling.
More in Models & Labs
© Hugging Face BlogModels & Labsmodels
vLLM V1 Achieves Backend Parity with V0
The transition from vLLM V0 to V1 represents a major backend overhaul, prioritizing parity before modifying reinforcement learning objectives. By resolving issues such as processed rollout logprobs and runtime defaults, the vLLM team ensured that V1's outputs meet the expectations set by V0. This approach demonstrates the critical role of backend accuracy in preserving training integrity. With these adjustments, V1 now mirrors V0's behavior, creating a stable foundation for future enhancements in RL objectives without the complications of backend discrepancies.
Hugging Face Blog
© TechCrunch AI