Open Source
llama.cpp b10046 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands support for AMD GPUs with ROCm 7.2, improving performance.
- →Ensures compatibility with the latest NVIDIA technologies through CUDA 12 and 13 DLLs.
- →Continues to broaden platform compatibility, making it a versatile tool for developers.
The b10046 release of llama.cpp introduces expanded support for multiple platforms, including Ubuntu with ROCm 7.2 and Windows with CUDA 12 and 13 DLLs. This update enhances compatibility and performance for both AMD and NVIDIA GPU users. While macOS Apple Silicon support is maintained, the KleidiAI feature is currently disabled. This release highlights llama.cpp's ongoing efforts to provide a versatile and comprehensive inference runtime for developers across various hardware configurations.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b10043 release supports CUDA Virtual Devices
The b10043 release of llama.cpp marks a notable enhancement with the addition of CUDA Virtual Devices, which significantly improves GPU resource management. By removing the NCCL path when virtual devices are in use, the update fine-tunes performance for these specific setups. This release also includes a comprehensive code refactor and the implementation of GPUx2 server CI jobs, reflecting a commitment to better testing and deployment processes. While there are no new model architectures, the update enhances the platform's flexibility across various operating systems, making it more adaptable for developers working with a wide range of hardware configurations.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10045 Release Expands Platform Support
The b10045 release of llama.cpp focuses on broadening its platform compatibility, though it doesn't introduce major new features. This update notably includes Vulkan support for Ubuntu and Windows, alongside ROCm 7.2 for Ubuntu, enhancing GPU utilization options for developers. While KleidiAI support for macOS Apple Silicon remains disabled, the release still covers a wide array of operating systems and architectures, offering developers increased flexibility in deployment. This update solidifies llama.cpp's position as a versatile inference runtime across multiple systems, rather than delivering groundbreaking changes.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10047 Release Expands Platform Support
The b10047 release of llama.cpp marks another step in its mission to support diverse hardware environments, now extending compatibility to platforms like macOS, Linux, and Windows. This update brings Vulkan support to Ubuntu and Windows, enhancing their graphics processing capabilities. The addition of ROCm 7.2 for Ubuntu x64 is a significant move for AMD GPU users, offering improved performance. While the release doesn't feature new models, it continues to support CUDA for NVIDIA users, ensuring robust performance across different setups. This version focuses on making llama.cpp a more versatile tool for developers working with various hardware configurations.
llama.cpp Releases
More in Open Source
© Lev SelectorOpen Sourcemodels
Tencent Releases Hy3: Open MoE LLM
Tencent has released Hy3, an open-source mixture of experts (MoE) large language model.
Lev Selector
© TechCrunch AIOpen Sourceresearch
Hugging Face CEO Advocates for Open Source AI
Clem Delangue, CEO of Hugging Face, underscores the critical role of open source AI, comparing the platform to a GitHub for AI models and datasets. He observes that as companies expand, they often move from expensive proprietary APIs to more affordable open source options, which he believes is essential for democratizing AI technology. Delangue voices concerns about the risk of a few large companies dominating the AI landscape, advocating for openness and transparency, particularly in the field of robotics. This approach is reflected in Hugging Face's decision to focus on capital efficiency rather than traditional fundraising, even declining a significant investment offer from Nvidia to stay true to its open source principles.
TechCrunch AI
© Hugging Face BlogOpen Sourceagents
NVIDIA's Nemotron Enhances AI with Open Synthetic Data
NVIDIA's Nemotron initiative is making strides in AI development by leveraging open synthetic data to enhance agent behavior and reproducibility. By releasing over 10 trillion pre-training tokens and millions of post-training samples, NVIDIA aims to make AI agents more adaptable and inspectable. The Nemotron Post-Training v3 Prompt Atlas offers an interactive way to explore this data, helping developers understand and refine model behaviors. This approach not only preserves proprietary data but also fosters a collaborative AI ecosystem, allowing diverse contributors to improve AI systems without compromising sensitive information.
Hugging Face Blog