Models & Labs
llama.cpp b11080 release with ROCm 10 and CUDA 13
llama.cpp Releaseshigh confidence
Why it matters
- →ROCm 10.0 support ensures AMD GPU users stay compatible with recent driver updates.
- →CUDA 13.4 compatibility prepares the library for NVIDIA's latest software stack releases.
- →Regex-based test filtering improves developer efficiency when debugging backend operations.
llama.cpp has released build b11080, adding support for ROCm 10.0 and CUDA 13.4 libraries on Linux and Windows platforms. The update also includes test suite improvements, specifically allowing regex entries in the -o filter for backend operations to streamline debugging. Existing builds for Apple Silicon, Vulkan, and OpenVINO remain available, with KleidiAI on macOS currently disabled in this release.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp adds env var control for server args
This release solves a real operational headache: configuring llama-server parameters like temperature and penalties via environment variables instead of command-line flags. By supporting LLAMA_ARG_* prefixes, it enables clean integration with systemd EnvironmentFile directives, making deployment on Linux servers significantly more robust and reproducible. The update also refreshes documentation using llama-gen-docs to keep the readme accurate. While no new inference engines are added, this is a practical step toward treating local LLM serving as standard infrastructure rather than a manual script.
llama.cpp Releases
Models & Labsother
llama.cpp b11081 release with CUDA 13 and ROCm 10 support
This release quietly cements llama.cpp as the universal inference runtime by adding default builds for CUDA 13.4 and ROCm 10.0, effectively closing the gap on newer NVIDIA and AMD hardware without requiring manual compilation flags. The inclusion of KleidiAI for Apple Silicon remains a key differentiator for local Mac users seeking optimized ARM kernels. While the changelog details internal test improvements like configurable tensor standard deviation, the real value lies in the expanded binary matrix that supports developers across the latest GPU architectures. Readers can now deploy on cutting-edge hardware out of the box. The update ensures that teams using the newest graphics cards do not need to wait for source code patches to achieve stable performance. This is a practical step toward making local inference truly hardware-agnostic.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11090 fixes CUDA Volta build
This release quietly repairs a regression in the CUDA backend that broke compilation on older Volta GPUs. By generalizing the tile shape logic in the load_ldmatrix kernel, the maintainers restored support for devices like the Tesla V100 without sacrificing newer architectures. It is a necessary patch for anyone running inference on legacy hardware, ensuring the codebase remains usable across the full spectrum of NVIDIA cards rather than drifting toward only the latest silicon.
llama.cpp Releases
More in Models & Labs
Models & Labsmodels
vLLM v0.30.0: DeepSeek V4.1 and Fast Start
This release is a massive infrastructure overhaul for serving the latest reasoning models. The headline feature is native support for DeepSeek-V4.1-Flash, storing its entire KV cache in MXFP8 on SM100 hardware to drastically reduce memory overhead. For operators tired of slow cold starts, the new Fast Start daemon caches post-quantized weights in GPU memory, allowing engines to map over CUDA IPC instead of reloading from disk. It also brings HiSparse, a host-resident tier that spills KV pages to pinned host memory under pressure, effectively expanding usable context windows without buying more GPUs.
vLLM Releases
© Duncan RogoffModels & Labsother
Jev AI Classifier: Speed vs. Hype
Jev positions itself as a specialized classifier claiming massive speed and cost advantages over frontier models like Opus and Fable. The reality is nuanced: it excels at rigid schema-based routing tasks but fails completely on open-ended generation or complex reasoning. This isn't a replacement for general-purpose LLMs, but rather a high-throughput router that can drastically cut inference costs when used correctly. The verdict suggests it's a useful tool for specific architectural patterns, not a magic bullet for all AI workloads.
Duncan Rogoff
© The AI Daily BriefModels & Labsresearch
Anthropic Opens New Biology Lab
Anthropic has established a new laboratory dedicated to biological research and AI applications in life sciences.
The AI Daily Brief