Models & Labs
llama.cpp b11007 enables CUDA graphs for MTP
llama.cpp Releaseshigh confidence
Why it matters
- →Enables CUDA graphs for MTP, reducing overhead and speeding up speculative decoding.
- →Updates CUDA binaries to support versions 12.8 and 13.3 for newer GPU drivers.
- →Maintains ROCm 10.0 support for AMD GPU users running local inference.
llama.cpp has released version b11007, introducing CUDA graph support for Multi-Token Prediction (MTP) to optimize speculative decoding performance. The update includes pre-built binaries for CUDA 12.8 and 13.3 across Linux and Windows, alongside ROCm 10.0 builds. Notably, KleidiAI acceleration on macOS Apple Silicon is currently disabled in this release. This patch targets developers running local LLMs who rely on draft models to accelerate inference speeds.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp b11000 fixes critical RCE in server
This release patches a critical remote code execution vulnerability in the llama.cpp server that allowed unauthenticated attackers to hijack memory via dangling pointers. The flaw stemmed from caching compute graphs that referenced freed buffers, enabling heap corruption and arbitrary code execution through subsequent tensor commands. By discarding cached graphs when buffers are freed, the fix forces a safe fallback to full recomputation without changing the API. This is a vital security update for anyone running the llama.cpp server remotely, closing a direct path to system compromise.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11001 fixes Spacemit transpose bug
A copy-paste error in llama.cpp was corrupting matrix transpositions on Spacemit hardware, causing significant data corruption for int16 operations. This release patches the specific RVV instruction call to ensure correct computation on these RISC-V based chips. While niche, it prevents silent inference failures for users relying on this specific accelerator architecture. The update also ships binaries for CUDA 13 and ROCm 10.0, keeping the runtime current with latest driver ecosystems.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11002 adds CUDA 13 and ROCm 10 support
This release quietly closes the hardware gap for local inference by adding native builds for CUDA 13 and ROCm 10.0 alongside existing CUDA 12 binaries. Users with newer NVIDIA GPUs or AMD accelerators no longer need to compile from source to get hardware acceleration, while Apple Silicon support remains available but explicitly disabled in this specific binary set. The simultaneous availability of these distinct backend versions means the runtime is now truly agnostic to the underlying GPU architecture. It is a pragmatic update that removes friction for anyone trying to run models on modern, non-CUDA-12 hardware.
llama.cpp Releases
More in Models & Labs
© TechCrunch AIModels & Labsother
Meta develops camera-free AI glasses 'Luna'
Meta is quietly addressing the privacy backlash against its Ray-Ban smart glasses by developing a new model called Luna that strips away cameras entirely. This device relies on six microphones and a side button to interact with Meta's AI chatbot and Muse agent, positioning audio-first interaction as a viable alternative to visual surveillance. By decoupling AI assistance from constant recording, Meta is testing whether users will adopt wearable AI if the social stigma of being watched is removed. It signals a strategic pivot toward privacy-preserving hardware to broaden adoption beyond early adopters comfortable with always-on cameras.
TechCrunch AI
© The Verge AIModels & Labsproductivity
Anthropic launches Claude Docs and Slides
Anthropic is finally closing the gap with Google by launching native document and presentation tools directly inside Claude. This isn't just a chat wrapper; it's a collaborative workspace where users can generate, edit, and share files in real-time, mirroring the utility of Google Docs and Slides. By merging these capabilities into a single interface alongside Artifacts and Design, Anthropic is transforming Claude from a conversational assistant into a comprehensive productivity platform. This move directly challenges Gemini's entrenched position in enterprise workflows by offering comparable creation tools without forcing users to leave the Anthropic ecosystem.
The Verge AI
© The Rundown AIModels & Labsagents
TypeSafe launches Jev: non-generative AI for software
TypeSafe’s new model Jev abandons text generation entirely to become a deterministic decision engine inside applications. By restricting outputs to pre-defined options with confidence scores, it claims zero hallucinations and runs at $42 per billion input tokens—roughly 238 times cheaper than leading frontier models. This approach targets high-volume internal logic like request sorting or safety screening rather than creative tasks. It represents a distinct shift toward specialized, cost-efficient AI infrastructure for enterprise software.
The Rundown AI