Models & Labs
OpenAI unveils Jalapeño AI chip, outperforms Nvidia
The Rundown AIhigh confidence
Why it matters
- →OpenAI's Jalapeño chip demonstrates significant advancements in AI hardware efficiency.
- →The chip's performance could lead to more cost-effective and faster AI model deployment.
- →This move positions OpenAI alongside other tech giants developing proprietary AI hardware.
OpenAI has introduced its first in-house AI chip, Jalapeño, which has shown to outperform Nvidia's top GPUs in speed and power efficiency. The chip, developed with Broadcom, is designed for running AI models and not for training them. In tests, Jalapeño operated at 700 watts, delivering up to 3.6 times faster performance and 1.9 times more work per watt compared to Nvidia's 1,200-watt options. OpenAI plans to use the chip internally, with no current plans to sell it, as it continues to rely on Nvidia for training new models. This development marks a significant step in OpenAI's hardware capabilities.
Read originalMore from The Rundown AI
More in Models & Labs
Models & Labsmodels
vLLM v0.28.0 Release: Major Performance Enhancements
The v0.28.0 release of vLLM introduces substantial improvements in performance and functionality, particularly for the Kimi-K3 model. With the addition of Decode Context Parallel support and fused FlashKDA decode kernels, the update significantly enhances processing speed and efficiency. DeepSeek V4 now includes sparse MLA support and advances in speculative decoding, offering better execution on both NVIDIA and AMD hardware. These updates make vLLM more robust and adaptable, providing developers with enhanced tools for deploying and executing models on a broader range of hardware configurations.
vLLM Releases
Models & Labsmodels
llama.cpp b10657 Release Expands Platform Support
The b10657 release of llama.cpp brings new OpenCL binary kernels, enhancing performance and compatibility across a wide range of systems. This update includes specific improvements for Apple Silicon, with KleidiAI support, and Vulkan on Ubuntu, making it more accessible for developers using these platforms. While no new model architectures are introduced, the release focuses on strengthening llama.cpp's capabilities as an inference runtime, particularly for those not using NVIDIA hardware. With ROCm 7.14 support on Ubuntu and CUDA 12 and 13 DLLs for Windows, llama.cpp continues to evolve as a versatile tool for AI model deployment. This release underscores the commitment to broadening hardware compatibility and optimizing performance across different environments.
llama.cpp Releases
Models & Labsmodels
llama.cpp b10658 release adds DFlash2 support
The b10658 release of llama.cpp marks a significant enhancement with the addition of DFlash2, which boosts local convolution and candidate selection capabilities. This update, with contributions from Claude Opus 5, focuses on optimizing costs and refining the code structure for better performance and maintainability. It also resolves several bugs and formatting issues, ensuring a more stable runtime. These improvements make llama.cpp more robust and efficient, catering to developers across various platforms. The release continues to solidify llama.cpp's position as a versatile tool for AI development.
llama.cpp Releases