Models & Labs
llama.cpp v0.6.0 adds decision model API and Apple Silicon speedups
llama.cpp Releaseshigh confidence
Why it matters
- →Enables local inference of decision models (Clef, GLM-5.3) via a new standardized API.
- →Adds mixed token/embedding batch support required for complex MTP architectures.
- →Delivers up to 3x speedup on Apple Silicon for speculative decoding workflows.
llama.cpp has released version 0.6.0, introducing support for decision models and multimodal inputs. Key additions include the /v1/systemone API endpoint for models like Clef and GLM-5.3-Flash, and a new llama_batch_ext API enabling mixed token/embedding batches. Performance improvements on Apple Silicon include new Metal MMA kernels offering up to 3x faster mat-mul operations. The release also updates ggml to v0.26.0 with sparse flash attention for Vulkan and CUDA optimizations.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
How we got here
- 1llama.cpp b10970 Release Expands Platform Support
llama.cpp Releases · September 15, 2026 · Same story
- 2llama.cpp b11081 release with CUDA 13 and ROCm 10 support
llama.cpp Releases · September 22, 2026 · Same story
More from llama.cpp Releases
Coding Toolscoding
llama.cpp b11424 adds CUDA 13 and ROCm 10.0
This release quietly closes the hardware gap for local inference by adding default support for CUDA 13 and ROCm 10.0 alongside existing CUDA 12 builds. NVIDIA users can now leverage newer driver stacks without manual configuration, while AMD GPU owners finally get first-class parity with the same ease of use previously reserved for CUDA. Apple Silicon KleidiAI is disabled in this specific build, a notable regression for Mac users who rely on that optimization. The inclusion of Snapdragon and OpenVINO binaries further broadens the reach to edge devices and Intel hardware. It’s less about new features and more about llama.cpp solidifying its position as the universal runtime for every major accelerator.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11425 adds ROCm 10 and Snapdragon support
This release quietly cements llama.cpp as the universal inference runtime by finally bringing first-class ROCm 10.0 support to both Linux and Windows. AMD GPU users no longer need workarounds, effectively closing a long-standing parity gap with NVIDIA's CUDA ecosystem. The inclusion of Snapdragon AI stack binaries for Linux marks a strategic push into ARM-based edge devices, while the simultaneous addition of CUDA 13 builds ensures compatibility with the latest driver stacks. By standardizing these hardware backends across major operating systems, the project removes friction for developers deploying models on diverse non-NVIDIA hardware.
llama.cpp Releases
Coding Toolscoding
llama.cpp 0.6.0 adds CUDA 13 and Snapdragon support
The llama.cpp 0.6.0 release quietly expands hardware coverage where it counts most: next-gen NVIDIA GPUs and mobile silicon. By shipping native builds for CUDA 13.4 alongside the existing CUDA 12 binaries, users can finally leverage newer GPU architectures without compiling from source. The inclusion of Linux arm64 support for Snapdragon chips with Adreno GPU and Hexagon NPU acceleration signals a serious push into on-device inference beyond Apple Silicon. While KleidiAI on macOS is temporarily disabled, the broader platform expansion makes this one of the most versatile local inference releases in recent memory.
llama.cpp Releases
More in Models & Labs
Models & Labscoding
vLLM v0.31.0: SM100 defaults and fast restarts
vLLM is quietly becoming the definitive runtime for NVIDIA's latest hardware, making NVFP4 compressed KV caches the default for DeepSeek-V4.1-Flash on SM100 GPUs. This isn't just a performance tweak; it fundamentally changes how enterprise inference scales by keeping post-quantized weights resident in GPU memory across engine restarts via the new preload daemon. The release also hardens speculative decoding with Model Runner V2, fixing OOMs that previously plagued wide expert deployments. For builders, this means lower latency and higher throughput on next-gen hardware without manual configuration overhead.
vLLM Releases
© Hugging Face BlogModels & Labsmodels
Falcon-Emirati-7B targets dialect nuance
Most Arabic models treat the language as a monolith, missing the cultural and linguistic depth of specific dialects. Falcon-Emirati-7B closes this gap by fine-tuning on native Emirati text, synthetic data constrained by strict glossaries, and cultural heritage knowledge. It tops the new Alyah benchmark with 84.83%, proving that scale alone doesn't buy dialect competence. This release underscores a critical shift: true multilingual capability requires targeted adaptation, not just larger parameter counts.
Hugging Face Blog
© TechCrunch AIModels & Labsmodels
Reflection debuts Beam open-weight model
Reflection AI is challenging the Chinese dominance in open-weight models with Beam, a 501B-parameter MoE model that claims to match Z.ai’s GLM-5.2 on reasoning benchmarks while using significantly less inference compute. Backed by $4.7 billion and secured GPU deals worth over $7 billion, this two-year-old startup is positioning itself as the Western alternative to DeepSeek and Qwen for enterprise and sovereign AI deployments. The model targets developers and institutions needing cost-effective, localizable infrastructure rather than just raw API access. With weights releasing this month, Beam offers a tangible option for those looking to reduce reliance on closed labs or Chinese open-source ecosystems.
TechCrunch AI