Models & Labs
Llama.cpp b10089 Release Enhances CUDA Support
llama.cpp Releaseshigh confidence
Why it matters
- →Enhances CUDA's efficiency in handling quantized data.
- →Reduces fallback to host, improving performance in single-device graphs.
- →Ensures comprehensive support for all quantized GGML types.
Llama.cpp's b10089 release introduces significant improvements to CUDA support, particularly in handling quantized data. The update adds k-quant and i-quant support to the GET_ROWS function, enabling more efficient device-side embedding lookups. This reduces the need for fallback to the host, enhancing performance in single-device graphs. The release also refines the handling of super-block dequantizers, ensuring comprehensive coverage for all quantized GGML types. These enhancements make CUDA more robust and efficient in processing quantized data.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
What happened next
- 1Llama.cpp b10099 Release Enhances CUDA Quantization
llama.cpp Releases · July 25, 2026 · Same story
- 2llama.cpp b10984 release enhances CUDA support
llama.cpp Releases · September 16, 2026 · Same story
More from llama.cpp Releases
Coding Toolscoding
llama.cpp b11183 splits Metal FA kernels
This release quietly refactors how llama.cpp handles Flash Attention on Apple Silicon by splitting kernels into per-dtype libraries. It’s a structural optimization that likely reduces memory overhead and improves compilation times for Metal users, though the immediate performance gains are subtle compared to algorithmic leaps. The build matrix remains massive, adding ROCm 10.0 and CUDA 13.4 support while disabling KleidiAI on Apple Silicon for now. This is infrastructure maintenance rather than a feature breakthrough, but it keeps the runtime robust across the expanding landscape of hardware backends.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11184 expands Metal support to wider blocks
This release tackles a specific performance bottleneck on Apple Silicon by extending Metal FWHT kernels to handle block widths up to 8192. Previously limited to 512, these wider operations now use threadgroup memory instead of registers, enabling faster inference for larger context windows or model architectures that rely on Hadamard transforms. The update also ships binaries for CUDA 13 and ROCm 10.0, keeping the toolkit aligned with the latest NVIDIA and AMD driver ecosystems. It’s a quiet but necessary optimization that improves throughput on M-series chips without changing the user experience.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11185 adds ROCm 10 and CUDA 13
One of llama.cpp's quietest but biggest releases. With ROCm 10.0 added as a default build, AMD GPU users stop being second-class citizens for local inference — the gap with CUDA narrows visibly. Apple Silicon Macs now compile in KleidiAI by default, meaning every M-series machine gets ARM-tuned GEMM kernels for free, no flag-flipping required. There's no new model and no new quantization here — just llama.cpp quietly becoming the inference runtime for everyone who isn't on NVIDIA. That's the headline.
llama.cpp Releases
More in Models & Labs
© Lev SelectorModels & Labsmodels
NVIDIA NVFP4 and SoL-Pi Token Efficiency
NVIDIA introduced the NVFP4 4-bit format and SoL-Pi technology, which uses 2x fewer tokens for improved efficiency.
Lev Selector
© Lev SelectorModels & Labsmodels
Claude Opus 5.5 and GPT-6 Sol Released
Anthropic released Claude Opus 5.5 on September 22, while OpenAI launched GPT-6 Sol and Luna variants, advancing frontier model capabilities.
Lev Selector
© TechCrunch AIModels & Labsresearch
GPT-6 Astra and Claude Opus 5 crack unsolved Enigma codes
OpenAI’s GPT-6 Astra and Anthropic’s Claude Opus 5 have independently broken long-standing Enigma ciphers that human cryptanalysts failed to solve for nearly two decades. This isn't just pattern matching; the models performed archival research, built simulators, and leveraged contextual clues to recover plaintext from messages dating back to 2005. The achievement demonstrates a leap in autonomous reasoning and tool use, effectively turning LLMs into professional researchers capable of multi-step problem solving that previously required weeks of human effort. It marks a significant shift in what we expect from frontier models beyond simple text generation.
TechCrunch AI