Models & Labs
llama.cpp b11081 release with CUDA 13 and ROCm 10 support
llama.cpp Releaseshigh confidence
Why it matters
- →CUDA 13.4 support enables immediate compatibility with the latest NVIDIA GPU drivers and hardware.
- →ROCm 10.0 integration provides native performance optimization for modern AMD accelerators.
- →Pre-built binaries reduce friction for developers who cannot or prefer not to compile from source.
llama.cpp has released version b11081, expanding its pre-built binary support to include CUDA 13.4 and ROCm 10.0 for both Linux and Windows platforms. The update also maintains compatibility with existing CUDA 12 builds and continues to feature KleidiAI optimization for Apple Silicon Macs. Internal testing infrastructure has been refined to better handle tensor data standard deviation and recurrent state rollback checks, ensuring stability across diverse model architectures. This release allows developers to run local inference on the latest NVIDIA and AMD GPUs without compiling from source.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp adds env var control for server args
This release solves a real operational headache: configuring llama-server parameters like temperature and penalties via environment variables instead of command-line flags. By supporting LLAMA_ARG_* prefixes, it enables clean integration with systemd EnvironmentFile directives, making deployment on Linux servers significantly more robust and reproducible. The update also refreshes documentation using llama-gen-docs to keep the readme accurate. While no new inference engines are added, this is a practical step toward treating local LLM serving as standard infrastructure rather than a manual script.
llama.cpp Releases
Models & Labsother
llama.cpp b11080 release with ROCm 10 and CUDA 13
This update quietly extends llama.cpp’s hardware support to ROCm 10.0 and CUDA 13.4 across Linux and Windows, keeping the library competitive as NVIDIA pushes newer driver stacks. The test suite also gains regex filtering for backend operations, a practical improvement for developers debugging specific inference paths. While no new model architectures are introduced, this release ensures compatibility with the latest GPU ecosystems without forcing users to wait for major version bumps.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11090 fixes CUDA Volta build
This release quietly repairs a regression in the CUDA backend that broke compilation on older Volta GPUs. By generalizing the tile shape logic in the load_ldmatrix kernel, the maintainers restored support for devices like the Tesla V100 without sacrificing newer architectures. It is a necessary patch for anyone running inference on legacy hardware, ensuring the codebase remains usable across the full spectrum of NVIDIA cards rather than drifting toward only the latest silicon.
llama.cpp Releases
More in Models & Labs
Models & Labsmodels
vLLM v0.30.0: DeepSeek V4.1 and Fast Start
This release is a massive infrastructure overhaul for serving the latest reasoning models. The headline feature is native support for DeepSeek-V4.1-Flash, storing its entire KV cache in MXFP8 on SM100 hardware to drastically reduce memory overhead. For operators tired of slow cold starts, the new Fast Start daemon caches post-quantized weights in GPU memory, allowing engines to map over CUDA IPC instead of reloading from disk. It also brings HiSparse, a host-resident tier that spills KV pages to pinned host memory under pressure, effectively expanding usable context windows without buying more GPUs.
vLLM Releases
© Duncan RogoffModels & Labsother
Jev AI Classifier: Speed vs. Hype
Jev positions itself as a specialized classifier claiming massive speed and cost advantages over frontier models like Opus and Fable. The reality is nuanced: it excels at rigid schema-based routing tasks but fails completely on open-ended generation or complex reasoning. This isn't a replacement for general-purpose LLMs, but rather a high-throughput router that can drastically cut inference costs when used correctly. The verdict suggests it's a useful tool for specific architectural patterns, not a magic bullet for all AI workloads.
Duncan Rogoff
© The AI Daily BriefModels & Labsresearch
Anthropic Opens New Biology Lab
Anthropic has established a new laboratory dedicated to biological research and AI applications in life sciences.
The AI Daily Brief