Models & Labs
NVIDIA Vera CPU Challenges Intel and AMD
NVIDIA Bloghigh confidence
Why it matters
- →Vera CPU offers significant performance improvements for AI workloads.
- →It challenges the dominance of traditional x86 CPUs from Intel and AMD.
- →Vera's efficiency and memory performance set new standards for AI infrastructure.
NVIDIA has unveiled its Vera CPU, which is set to challenge the dominance of Intel and AMD in the CPU market. The Vera CPU, equipped with 88 custom Olympus cores, delivers exceptional performance for AI-centric workloads, boasting a memory bandwidth of 1.2TB/s. Initial benchmarks by Phoronix reveal that Vera outperforms traditional x86 CPUs in both power efficiency and memory performance. This marks a significant generational leap from NVIDIA's previous Grace CPU. With its upcoming availability through partners, Vera is poised to become a key player in AI infrastructure.
Read originalMore in Models & Labs
Models & Labsmodels
Llama.cpp b9329 Release Enhances CUDA Performance
The b9329 release of llama.cpp brings a notable performance enhancement with the integration of a fast Walsh-Hadamard transform for CUDA, which is set to improve computational efficiency. This update also includes optimizations such as unrolling and changes from size_t to int, aimed at boosting processing speed. The release is compatible with platforms like macOS, Linux, Windows, and openEuler, ensuring developers can leverage these improvements across different environments. While there are no new models introduced, the emphasis on performance optimization makes this update significant for those working with CUDA and other supported systems.
llama.cpp Releases
Models & Labsmodels
llama.cpp b9330 release improves model performance
The b9330 release of llama.cpp resolves a key issue by correctly tagging the ffn_latent operation as MUL_MAT, aligning it with the backend's operational expectations. This correction ensures that weights and their matrix multiplications remain on the GPU, avoiding unnecessary CPU fallback and graph splitting. As a result, performance on the Nemotron 3 Super 120B Q5_K_M model has significantly improved, with throughput increasing from 64.9 to 103.22 tokens per second. This update reflects llama.cpp's dedication to enhancing AI model performance across different computing environments, including macOS with KleidiAI and Ubuntu with ROCm 7.2. By maintaining efficient GPU processing, llama.cpp continues to optimize AI model execution, ensuring robust performance on platforms like CUDA 12 and CUDA 13.
llama.cpp Releases
Models & Labsmodels
llama.cpp b9334 Release Expands Platform Support
The latest b9334 release of llama.cpp significantly broadens its platform compatibility, making it more accessible to a diverse range of users. With new support for macOS Apple Silicon, Ubuntu with ROCm 7.2, and Windows with CUDA 12 and 13, this update ensures that developers across different systems can leverage llama.cpp's capabilities. The inclusion of Vulkan and SYCL support further enhances its versatility, catering to both CPU and GPU users. This release doesn't introduce new models but focuses on making llama.cpp a more universal tool for AI inference across various hardware configurations.
llama.cpp Releases