Models & Labs
NVIDIA Unveils Nemotron 3.5 Lightning and NeMo Switchyard
NVIDIA Bloghigh confidence
Why it matters
- →Nemotron 3.5 Lightning offers significant speed improvements for agentic AI tasks.
- →NeMo Switchyard optimizes model routing, reducing costs and improving efficiency.
- →These tools enhance control over AI deployment across various platforms.
NVIDIA has introduced Nemotron 3.5 Lightning, a high-efficiency model for agentic AI workloads, and NeMo Switchyard, an open-source library for smart model routing. Nemotron 3.5 Lightning, a 30-billion-parameter model, offers significant speed improvements and customization options for specialized tasks. NeMo Switchyard allows enterprises to route AI tasks to the most suitable models, optimizing performance and cost. These releases enhance AI deployment flexibility across different environments, from local systems to cloud platforms.
Read originalMore from NVIDIA Blog
© NVIDIA BlogMarket & Regulationbusiness
NVIDIA Partners to Finance AI Infrastructure
NVIDIA is redefining AI infrastructure as an investable asset by collaborating with financial powerhouses like BlackRock and Goldman Sachs. This strategic partnership is set to channel over $500 billion in third-party capital towards the creation of AI factories. These factories go beyond mere hardware, offering a complete platform with accelerated computing and AI frameworks, ensuring they remain adaptable and valuable over time. This initiative marks a pivotal change in AI infrastructure financing, enhancing its accessibility and sustainability for diverse industries and applications.
NVIDIA Blog
© NVIDIA BlogMarket & Regulationbusiness
NVIDIA Pushes 800 VDC for AI Compute Power
NVIDIA, in collaboration with Google and Microsoft, is spearheading a shift in AI compute power architecture with the introduction of 800 VDC. This new power distribution method aims to streamline the path from the grid to GPUs, reducing inefficiencies inherent in traditional AC systems. By adopting 800 VDC, AI factories can achieve higher compute performance and power density without overhauling existing infrastructure. This initiative is backed by over 80 companies and promises a scalable roadmap for AI facilities, ensuring they can meet future demands without stranding current investments.
NVIDIA Blog
© NVIDIA BlogOpen Sourceagents
NVIDIA Boosts Local AI with New Open Models
NVIDIA is making strides in the open-source AI ecosystem by releasing several new models and tools that enhance local AI capabilities. Notably, the Cosmos 3 Edge model for robotics and autonomous vehicles, and the MiniMax-H3 model for video and audio generation, are optimized for NVIDIA GPUs, allowing developers to run complex AI tasks locally. This push towards local AI is further supported by the launch of Unsloth Desktop, a comprehensive open-source app for AI model training and inference. These developments signify a shift towards more accessible and efficient AI processing on personal devices, reducing reliance on cloud-based solutions.
NVIDIA Blog
More in Models & Labs
Models & Labsmodels
llama.cpp b10412 Release Enhances Backend Sampling
The latest b10412 release of llama.cpp introduces backend sampling for both dflash and dspark, marking a technical enhancement in the platform's capabilities. This update allows for more refined control with the enablement of p_min > 0 in backend sampling, adding a layer of precision for developers. While the release doesn't introduce new models or architectures, it quietly strengthens the platform's backend functionality, making it more versatile for developers working across various systems. This update is a step forward in optimizing the performance and flexibility of llama.cpp's inference capabilities.
llama.cpp Releases
Models & Labsmodels
llama.cpp b10414 Release Adds TQ2_0 Support
The b10414 release of llama.cpp marks a significant enhancement with the addition of GGML_TYPE_TQ2_0 type processing in the Metal backend, enabling ternary operations with 2 bits per element. This update brings a more efficient mul_mv kernel, focusing on float operations and optimizing data handling through techniques like precalculating sums. While the release doesn't feature new models, it refines the platform's performance and broadens its compatibility across systems like macOS, Linux, and Windows. By improving efficiency and versatility, llama.cpp continues to be a valuable tool for developers working with a variety of hardware configurations.
llama.cpp Releases
Models & Labsmodels
llama.cpp b10418 Release Enhances SYCL Support
The b10418 release of llama.cpp brings notable improvements to SYCL support, particularly through the introduction of host pinned memory, which enhances host-to-device memory access. This update also resolves a thread-safety issue, ensuring more stable performance across different hardware setups. While no new models are introduced, the release focuses on strengthening the existing infrastructure, making it more robust for developers working with SYCL. This update is crucial for optimizing performance and ensuring compatibility, especially for those leveraging SYCL in their development environments.
llama.cpp Releases