Models & Labs
vLLM v0.26.0 Release Enhances Model Support
vLLM Releaseshigh confidence
Why it matters
- →The new Inkling model family introduces advanced features for improved AI model performance.
- →Enhanced hardware support broadens the applicability of vLLM across different platforms.
- →Improved attention mechanisms and KV offloading increase efficiency for hybrid models.
vLLM has released version 0.26.0, featuring significant updates including the new Inkling model family with advanced CUDA graph support and speculative decoding. The release also enhances performance across multiple hardware platforms, such as AMD and XPU, with the DeepSeek-V4 performance push. Improvements in attention mechanisms and KV offloading provide greater flexibility for hybrid models. These updates make vLLM a more powerful tool for developers handling large-scale AI models.
Read originalMore in Models & Labs
Models & Labsmodels
Llama.cpp Adds Vision Support with MiniMax-M3
Llama.cpp's latest update introduces preliminary support for the MiniMax-M3 model, marking a significant step towards integrating vision capabilities. This release reuses existing components from MiniMax-M2, incorporating advanced features like per-head QK-norm and partial rotary, while also optimizing performance with GPU and CPU operations. Although sparse attention isn't supported yet, the update promises a substantial speedup in processing long contexts. This development positions llama.cpp to better handle vision tasks, expanding its utility beyond text-only applications.
llama.cpp Releases
Models & Labsmodels
llama.cpp b10144 Release Fixes Stream Routes
The latest b10144 release of llama.cpp addresses several issues related to stream routes and model loading. Notably, it fixes problems with model names containing slashes, ensuring that stop and resume functions work correctly. The update also improves the handling of pending requests during model loading, allowing sessions to persist even if a page is reloaded. These changes enhance the reliability and user experience of the platform, particularly for developers working with complex model names and streaming data.
llama.cpp Releases
© NVIDIA BlogModels & Labsmodels
NVIDIA Uses Vera CPU to Enhance Chip Design
NVIDIA is deploying its Vera CPU to accelerate the design of its next-generation CPUs and GPUs, working with Cadence and Synopsys to optimize electronic design automation (EDA) applications. This initiative underscores the critical role of high-performance CPU architecture in expediting engineering workloads such as logic simulation and formal verification. Early testing reveals up to 1.5x performance improvements in key EDA applications, demonstrating Vera's potential to boost productivity in chip design. By integrating Vera into its workflows, NVIDIA aims to create a feedback loop that continuously refines its silicon design process, paving the way for more efficient future developments.
NVIDIA Blog