© Matt WolfeCoding Toolsagents
OpenAI Introduces GPT-Live and Agent API
OpenAI launches GPT-Live 1 and a new Agent API, enabling real-time voice interactions and autonomous agent development.
Matt Wolfe
The llama.cpp server has been updated to support chunked prefill for causal LLM rerankers, specifically addressing limitations with models like Qwen3 and Qwen3-VL. Previously, the system rejected RANK-pooling inputs exceeding n_ubatch due to hardcoded architectural checks, effectively breaking long-document and multimodal reranking tasks. The fix exposes runtime attention type checks, allowing the graph builder to dynamically permit chunked prefill when the context is causal. This change removes code duplication and enables efficient processing of larger batches for these specific model architectures.
Read originalEarlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
llama.cpp Releases · August 16, 2026 · Related
llama.cpp Releases · September 24, 2026 · Same story
This release quietly cements llama.cpp as the universal inference runtime by finally bringing first-class ROCm 10.0 support to both Linux and Windows. AMD GPU users no longer need workarounds for modern hardware, closing a long-standing parity gap with NVIDIA's CUDA ecosystem. Equally notable is the new Linux arm64 build targeting Snapdragon chips, which unlocks local LLM inference on high-performance mobile SoCs via CPU, Adreno GPU, and Hexagon NPU acceleration. While KleidiAI on Apple Silicon has been disabled in this specific binary set, the broader platform coverage signals a strategic shift toward hardware agnosticism that benefits anyone running models outside of standard NVIDIA setups.
This release quietly sharpens llama.cpp’s edge on AMD hardware by enabling the fattn-mma kernel for large query dimensions on CDNA architectures. It specifically targets high-throughput scenarios where batch sizes push dkq beyond 256, a common bottleneck in serving workloads. By optimizing these specific matrix multiplication paths, the update reduces latency and improves throughput for enterprise-style inference without requiring code changes. This is another step in making AMD GPUs competitive with NVIDIA for heavy lifting.
This release stabilizes Claude Code by fixing a cascade of session-breaking errors that previously caused silent data loss or API drops. The most significant fix addresses resumed conversations re-sending messages in altered forms, which was corrupting reasoning traces and breaking extended thinking workflows. It also resolves persistent login refresh loops and managed setting parsing failures that plagued enterprise deployments. While the changelog is dense with UI tweaks like scrollbar fixes and vim mode corrections, the core value lies in restoring reliability for long-running agent sessions.
This release quietly solves a major pain point for enterprise AI workflows by adding gateway hint headers, allowing LLM gateways to correctly group requests per user prompt instead of treating them as isolated events. The new managed settings for availableModelsMatch and deniedModels give organizations precise control over model access, blocking specific versions even when broader allowances exist. Beyond governance, the update stabilizes the plugin ecosystem with rigorous validation checks that prevent silent failures from broken or misconfigured extensions. These changes shift Claude Code from a developer tool to a manageable enterprise component.