Models & Labs
llama.cpp b10996 adds CUDA 13 and ROCm 10.0 builds
llama.cpp Releaseshigh confidence
Why it matters
- →CUDA 13 support allows users on newer NVIDIA drivers to run local models without compatibility workarounds.
- →ROCm 10.0 binaries give AMD GPU users parity with NVIDIA in terms of official release availability.
- →openEuler Ascend support expands viable hardware options for inference in specific enterprise or Chinese markets.
llama.cpp has released version b10996, expanding hardware compatibility with new builds for CUDA 13 and ROCm 10.0 on Linux and Windows. The update also adds support for openEuler on Huawei Ascend 910b chips via ACL Graph, while explicitly disabling KleidiAI optimizations for macOS Apple Silicon in this build. Existing binaries for CUDA 12, Vulkan, OpenVINO, and SYCL remain available alongside the new additions. This release primarily serves to keep the inference runtime aligned with the latest driver and library versions for NVIDIA and AMD GPUs.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp b10993 adds ROCm 10 and CUDA 13 builds
This release quietly extends llama.cpp's hardware support to the latest driver stacks, adding official binaries for ROCm 10.0 and CUDA 13 across Linux and Windows. For AMD users, this means native compatibility with newer GPU architectures without manual compilation tweaks, while NVIDIA users gain access to the latest CUDA runtime optimizations. The inclusion of WebGPU in CI signals ongoing work toward browser-based inference, though it remains a background effort for now. There are no new model formats or quantization methods here, just broader infrastructure coverage that keeps llama.cpp relevant as hardware evolves.
llama.cpp Releases
Coding Toolscoding
llama.cpp fixes Metal NaN bug for large activations
A critical precision bug in llama.cpp’s Apple Silicon backend has been patched, resolving total inference failures on models with high-activation ranges like Mistral Small 4. The issue stemmed from f16 saturation during matrix multiplication, which turned entire output tensors into NaN values for inputs exceeding ~32 tokens. By implementing an exact, power-of-two rescaling mechanism in the Metal kernel, the fix restores correctness without significant performance penalties. This ensures local inference on M-series chips remains viable for complex MoE architectures that previously crashed.
llama.cpp Releases
Coding Toolscoding
llama.cpp b10995 Vulkan and ROCm updates
This release quietly improves inference stability on AMD GPUs by unconditionally enabling the BN/2 tail for MUL_MAT_ID operations in Vulkan. It also adds ROCm 10.0 builds for both Linux and Windows, keeping llama.cpp competitive with CUDA across hardware vendors. Apple Silicon KleidiAI builds are disabled here, likely due to regression testing, but the core CPU and NVIDIA paths remain robust. For local inference users, this means fewer edge-case crashes on AMD setups without needing custom compilation flags.
llama.cpp Releases
More in Models & Labs
© TechCrunch AIModels & Labsmodels
Salesforce and Nvidia Launch Koa AI Model
Salesforce's launch of Koa, a reasoning model developed with Nvidia's Nemotron, represents a pivotal moment for enterprise AI. Koa is specifically designed for sales and customer support tasks, offering an open-weight model that prioritizes privacy by not utilizing customer data. This approach contrasts with the broader, more theoretical models from frontier labs, focusing instead on practical applications that reduce costs and enhance efficiency. By integrating Koa into its Agentforce platform, Salesforce provides a tailored alternative to models like Claude and ChatGPT, emphasizing secure and efficient AI solutions for businesses.
TechCrunch AI
Models & Labsmodels
Perplexity Adopts GPT-6 Astra for System Management
Perplexity has integrated GPT-6 Astra into its operations, marking a significant shift in how AI can manage complex systems. By entrusting Astra with tasks like writing communications, altering software, and monitoring production systems, Perplexity demonstrates a high level of confidence in the model's capabilities. This move reduces the need for frequent human oversight, suggesting that Astra's reliability and efficiency surpass previous models. The adoption of GPT-6 Astra could signal a new era where AI takes on more autonomous roles in managing end-to-end systems.
OpenAI
© The AI Daily BriefModels & Labsother
OpenAI Launches GPT Live Voice API
OpenAI has released a new voice API for real-time interaction with GPT models.
The AI Daily Brief