vLLM has released version 0.28.0, featuring substantial performance improvements and new capabilities. Key updates include major optimizations for the Kimi-K3 model, such as Decode Context Parallel support and fused FlashKDA decode kernels. The release also enhances DeepSeek V4 with sparse MLA support and speculative decoding, benefiting both NVIDIA and AMD hardware users. These advancements aim to improve model execution efficiency and expand vLLM's applicability across different hardware platforms.
Read originalThe b10657 release of llama.cpp brings new OpenCL binary kernels, enhancing performance and compatibility across a wide range of systems. This update includes specific improvements for Apple Silicon, with KleidiAI support, and Vulkan on Ubuntu, making it more accessible for developers using these platforms. While no new model architectures are introduced, the release focuses on strengthening llama.cpp's capabilities as an inference runtime, particularly for those not using NVIDIA hardware. With ROCm 7.14 support on Ubuntu and CUDA 12 and 13 DLLs for Windows, llama.cpp continues to evolve as a versatile tool for AI model deployment. This release underscores the commitment to broadening hardware compatibility and optimizing performance across different environments.
The b10658 release of llama.cpp marks a significant enhancement with the addition of DFlash2, which boosts local convolution and candidate selection capabilities. This update, with contributions from Claude Opus 5, focuses on optimizing costs and refining the code structure for better performance and maintainability. It also resolves several bugs and formatting issues, ensuring a more stable runtime. These improvements make llama.cpp more robust and efficient, catering to developers across various platforms. The release continues to solidify llama.cpp's position as a versatile tool for AI development.
The b10659 release of llama.cpp brings a crucial update for Windows users by including HIP runtime DLLs with the Windows ROCm package. This ensures that the correct HIP runtime is prioritized over the driver's version in System32, effectively solving a previous issue. Although this update doesn't introduce new model architectures or quantization techniques, it significantly enhances the platform's compatibility and performance. Developers working on AI tasks in Windows environments can now expect a more streamlined setup process and potentially better runtime performance.