Models & Labs
Llama.cpp b9133 Release Enhances Reasoning Models
llama.cpp Releaseshigh confidence
Why it matters
- →Enhances reasoning model capabilities in server and web UI environments.
- →Introduces mechanisms for smoother continuation of generation tasks.
- →Sets the groundwork for future improvements in reasoning models.
Llama.cpp has released its b9133 update, focusing on enhancing support for reasoning models in server and web UI contexts. The update removes blocking assistant prefill and introduces thinking tags to improve the continuation of generation tasks. It also allows reasoning content to persist through reloads by dropping the reasoning guard on the Continue button. This release is a step towards more advanced reasoning capabilities, although channel-based templates remain out of scope for now.
Read originalMore from llama.cpp Releases
Open Sourcecoding
llama.cpp b9129 Release Enhances CPU Fallback
The b9129 release of llama.cpp introduces an adaptive fallback feature for the ggml-zendnn backend, which optimizes performance by switching to the CPU for small batch sizes. This feature is enabled by default, but developers can control it using a new runtime environment variable, allowing them to revert to the original fallback logic if desired. The update supports platforms like macOS with KleidiAI, Windows with CUDA 12 and 13, and Ubuntu with ROCm 7.2, ensuring efficient processing across different systems. This release highlights llama.cpp's focus on enhancing performance and flexibility for developers working with various hardware configurations.
llama.cpp Releases
Open Sourcemodels
llama.cpp b9134 Release Expands Platform Support
The latest b9134 release of llama.cpp continues its trend of broadening platform compatibility, making it a versatile tool for developers across various systems. This update includes support for macOS Apple Silicon with KleidiAI enabled, as well as expanded Vulkan and ROCm 7.2 support on Ubuntu. Windows users benefit from updated CUDA 12 and 13 DLLs, enhancing performance for GPU tasks. While no new models are introduced, the release solidifies llama.cpp's position as a flexible inference runtime across diverse hardware configurations.
More in Models & Labs
© TechCrunch AIModels & Labsmodels
Anthropic's AI Vision: Anticipating User Needs
Anthropic is making waves in the AI industry with its proactive approach to AI development, aiming to create models that anticipate user needs before they even arise. Cat Wu, a key figure at Anthropic, emphasizes the importance of staying at the forefront of AI innovation without being reactive to competitors. The company's recent initiatives, like the Glasswing project, highlight its commitment to safe and impactful AI deployment. As Anthropic continues to expand its market share, the focus is on developing AI that can automate routine tasks, potentially transforming workplace productivity.
TechCrunch AI
© Microsoft Research