Coding Tools
llama.cpp fixes Metal NaN bug for large activations
llama.cpp Releaseshigh confidence
Why it matters
- →Fixes total inference failure on Apple Silicon for models with high activation ranges.
- →Enables local running of large MoE models like Mistral Small 4 without workarounds.
- →Maintains performance with minimal overhead (~1-4%) via exact rescaling.
llama.cpp has released a patch addressing a severe NaN propagation bug in its Metal backend for Apple Silicon devices. The defect occurred when activation values exceeded the f16 range (65504), causing matrix multiplication results to become entirely invalid, particularly affecting large Mixture-of-Experts models like Mistral Small 4. The fix implements an exact rescaling strategy within the kernel that maintains numerical stability while adding only ~1-4% latency overhead. This resolves long-standing issues where generation would fail completely for inputs longer than 32 tokens.
Read originalMore from llama.cpp Releases
Coding Toolscoding
llama.cpp b10993 adds ROCm 10 and CUDA 13 builds
This release quietly extends llama.cpp's hardware support to the latest driver stacks, adding official binaries for ROCm 10.0 and CUDA 13 across Linux and Windows. For AMD users, this means native compatibility with newer GPU architectures without manual compilation tweaks, while NVIDIA users gain access to the latest CUDA runtime optimizations. The inclusion of WebGPU in CI signals ongoing work toward browser-based inference, though it remains a background effort for now. There are no new model formats or quantization methods here, just broader infrastructure coverage that keeps llama.cpp relevant as hardware evolves.
llama.cpp Releases
Coding Toolscoding
llama.cpp b10995 Vulkan and ROCm updates
This release quietly improves inference stability on AMD GPUs by unconditionally enabling the BN/2 tail for MUL_MAT_ID operations in Vulkan. It also adds ROCm 10.0 builds for both Linux and Windows, keeping llama.cpp competitive with CUDA across hardware vendors. Apple Silicon KleidiAI builds are disabled here, likely due to regression testing, but the core CPU and NVIDIA paths remain robust. For local inference users, this means fewer edge-case crashes on AMD setups without needing custom compilation flags.
llama.cpp Releases
Models & Labscoding
llama.cpp b10996 adds CUDA 13 and ROCm 10.0 builds
This release quietly closes the hardware gap for local inference by adding native support for CUDA 13 and ROCm 10.0 across Linux and Windows. NVIDIA users can now leverage newer driver stacks without waiting for upstream updates, while AMD GPU owners finally get first-class ROCm 10 binaries that match the maturity of their CUDA counterparts. Apple Silicon builds remain available but KleidiAI is explicitly disabled here, suggesting a focus on stability over new kernel optimizations for this specific iteration. The inclusion of openEuler support for Huawei's Ascend chips further broadens the ecosystem beyond standard x86 and ARM consumer hardware.
llama.cpp Releases
More in Coding Tools
Coding Toolscoding
vLLM v0.29.0rc6 fixes hybrid model caching
This release candidate addresses a critical configuration gap for hybrid models in vLLM by enabling the dense prefix cache by default. Previously, these mixed-precision or multi-head architectures likely suffered from inefficient memory usage or required manual flags to achieve optimal performance. By automating this optimization, the update ensures that users running complex model topologies get better throughput out of the box without tweaking internal parameters. It is a quiet but necessary step toward making vLLM robust for heterogeneous hardware setups.
vLLM Releases
© GitHub ChangelogCoding Toolscoding
GitHub Copilot Enhances Custom Property Suggestions
GitHub Copilot is now assisting enterprise and organization admins by suggesting allowed values for custom properties in repositories. This feature, available in public preview for Copilot Business and Enterprise plans, tackles the issue of inconsistent metadata by offering relevant suggestions tailored to the property being defined. For example, when setting up a property like 'FedRAMP', Copilot proposes compliance-related values, making it easier to establish a consistent governance framework. This development accelerates the creation of custom property taxonomies, improving the application of governance rules across extensive repository collections.
GitHub Changelog
Coding Toolscoding
Claude Code v2.1.261 Update Released
The latest update to Claude Code, version 2.1.261, introduces several enhancements and fixes aimed at improving user experience and functionality. Notable changes include the addition of settings to increase command output limits and a new feature to identify unused skills, helping users optimize their setups. The update also addresses various bugs, such as character input issues and session management glitches, ensuring smoother operation across different environments. These improvements make Claude Code more robust and user-friendly, particularly for developers using it in complex workflows.
Claude Code Releases