Open Source
llama.cpp b10045 Release Expands Platform Support
llama.cpp Releaseshigh confidence
Why it matters
- →Expands platform support, enhancing deployment flexibility for developers.
- →Includes Vulkan and ROCm 7.2 support, improving GPU utilization options.
- →Reinforces llama.cpp's role as a versatile inference runtime.
The b10045 release of llama.cpp has been announced, focusing on expanding platform support rather than introducing new features. This update includes Vulkan support for both Ubuntu and Windows, and ROCm 7.2 for Ubuntu, providing developers with more GPU options. Although KleidiAI support for macOS Apple Silicon is disabled, the release maintains a wide range of compatibility across various operating systems and architectures. This positions llama.cpp as a versatile tool for developers looking to deploy AI models across different environments.
Read originalMore from llama.cpp Releases
Models & Labsmodels
llama.cpp b10043 release supports CUDA Virtual Devices
The b10043 release of llama.cpp marks a notable enhancement with the addition of CUDA Virtual Devices, which significantly improves GPU resource management. By removing the NCCL path when virtual devices are in use, the update fine-tunes performance for these specific setups. This release also includes a comprehensive code refactor and the implementation of GPUx2 server CI jobs, reflecting a commitment to better testing and deployment processes. While there are no new model architectures, the update enhances the platform's flexibility across various operating systems, making it more adaptable for developers working with a wide range of hardware configurations.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10046 Release Expands Platform Support
The b10046 release of llama.cpp continues to broaden its platform compatibility, making it an adaptable tool for developers working across different systems. This update notably includes support for Ubuntu with ROCm 7.2, which enhances performance for AMD GPU users. Windows users gain from the inclusion of CUDA 12 and 13 DLLs, ensuring they can leverage the latest NVIDIA technologies. While macOS Apple Silicon support remains strong, the KleidiAI feature is temporarily disabled. This release reflects llama.cpp's ongoing effort to be a comprehensive inference runtime across a wide range of hardware configurations.
llama.cpp Releases
Open Sourcemodels
llama.cpp b10047 Release Expands Platform Support
The b10047 release of llama.cpp marks another step in its mission to support diverse hardware environments, now extending compatibility to platforms like macOS, Linux, and Windows. This update brings Vulkan support to Ubuntu and Windows, enhancing their graphics processing capabilities. The addition of ROCm 7.2 for Ubuntu x64 is a significant move for AMD GPU users, offering improved performance. While the release doesn't feature new models, it continues to support CUDA for NVIDIA users, ensuring robust performance across different setups. This version focuses on making llama.cpp a more versatile tool for developers working with various hardware configurations.
llama.cpp Releases
More in Open Source
© Lev SelectorOpen Sourcemodels
Tencent Releases Hy3: Open MoE LLM
Tencent has released Hy3, an open-source mixture of experts (MoE) large language model.
Lev Selector
© TechCrunch AIOpen Sourceresearch
Hugging Face CEO Advocates for Open Source AI
Clem Delangue, CEO of Hugging Face, underscores the critical role of open source AI, comparing the platform to a GitHub for AI models and datasets. He observes that as companies expand, they often move from expensive proprietary APIs to more affordable open source options, which he believes is essential for democratizing AI technology. Delangue voices concerns about the risk of a few large companies dominating the AI landscape, advocating for openness and transparency, particularly in the field of robotics. This approach is reflected in Hugging Face's decision to focus on capital efficiency rather than traditional fundraising, even declining a significant investment offer from Nvidia to stay true to its open source principles.
TechCrunch AI
© Hugging Face BlogOpen Sourceagents
NVIDIA's Nemotron Enhances AI with Open Synthetic Data
NVIDIA's Nemotron initiative is making strides in AI development by leveraging open synthetic data to enhance agent behavior and reproducibility. By releasing over 10 trillion pre-training tokens and millions of post-training samples, NVIDIA aims to make AI agents more adaptable and inspectable. The Nemotron Post-Training v3 Prompt Atlas offers an interactive way to explore this data, helping developers understand and refine model behaviors. This approach not only preserves proprietary data but also fosters a collaborative AI ecosystem, allowing diverse contributors to improve AI systems without compromising sensitive information.
Hugging Face Blog