Models & Labs
NVIDIA Expands NVLink Fusion with NVHBM
NVIDIA Bloghigh confidence
Why it matters
- →NVHBM increases memory bandwidth and reduces power consumption, enhancing AI performance.
- →The integration simplifies the engineering process for custom AI chip development.
- →Collaboration with Amazon's Annapurna Labs highlights industry adoption and potential impact.
NVIDIA has expanded its NVLink Fusion technology with the introduction of NVHBM, a new high-bandwidth memory solution. NVHBM integrates the memory controller into the HBM stack, increasing memory bandwidth by 30% and reducing power consumption by 15%. This advancement allows for more efficient use of silicon area on XPUs. Amazon's Annapurna Labs will be the first to utilize NVHBM, as part of its collaboration with NVIDIA. This move is expected to streamline the development of custom AI chips, enhancing performance and efficiency for AI workloads.
Read originalMore from NVIDIA Blog
More in Models & Labs
Models & Labsmodels
vLLM v0.28.0 Release: Major Performance Enhancements
The v0.28.0 release of vLLM introduces substantial improvements in performance and functionality, particularly for the Kimi-K3 model. With the addition of Decode Context Parallel support and fused FlashKDA decode kernels, the update significantly enhances processing speed and efficiency. DeepSeek V4 now includes sparse MLA support and advances in speculative decoding, offering better execution on both NVIDIA and AMD hardware. These updates make vLLM more robust and adaptable, providing developers with enhanced tools for deploying and executing models on a broader range of hardware configurations.
vLLM Releases
Models & Labsmodels
llama.cpp b10657 Release Expands Platform Support
The b10657 release of llama.cpp brings new OpenCL binary kernels, enhancing performance and compatibility across a wide range of systems. This update includes specific improvements for Apple Silicon, with KleidiAI support, and Vulkan on Ubuntu, making it more accessible for developers using these platforms. While no new model architectures are introduced, the release focuses on strengthening llama.cpp's capabilities as an inference runtime, particularly for those not using NVIDIA hardware. With ROCm 7.14 support on Ubuntu and CUDA 12 and 13 DLLs for Windows, llama.cpp continues to evolve as a versatile tool for AI model deployment. This release underscores the commitment to broadening hardware compatibility and optimizing performance across different environments.