Models & Labs

Open ASR Leaderboard Adds Private Datasets

Hugging Face BlogMay 6, 2026high confidence

Why it matters

→Incorporating private datasets reduces the risk of benchmaxxing, leading to more reliable ASR evaluations.
→The update allows for a more comprehensive assessment of ASR models across diverse accents and speech types.
→It balances the need for openness with the necessity of robust, real-world performance metrics.

Open ASR Leaderboard Adds Private Datasets — ©Hugging Face Blog

The Open ASR Leaderboard has introduced private datasets from Appen Inc. and DataoceanAI to enhance its benchmarking process. These datasets, which include a variety of accents and speech types, are intended to prevent benchmaxxing and improve the accuracy of ASR performance evaluations. The leaderboard's average Word Error Rate (WER) will continue to be calculated using public datasets by default, but users can choose to include private datasets for a more detailed analysis. This update aims to provide a more nuanced view of ASR model performance across different conditions.

Read original

More from Hugging Face Blog

Models & Labsmodels

vLLM V1 Achieves Backend Parity with V0

The transition from vLLM V0 to V1 represents a major backend overhaul, prioritizing parity before modifying reinforcement learning objectives. By resolving issues such as processed rollout logprobs and runtime defaults, the vLLM team ensured that V1's outputs meet the expectations set by V0. This approach demonstrates the critical role of backend accuracy in preserving training integrity. With these adjustments, V1 now mirrors V0's behavior, creating a stable foundation for future enhancements in RL objectives without the complications of backend discrepancies.

Hugging Face BlogMay 6, 2026

More in Models & Labs

Models & Labsmodels

llama.cpp b9041 Release Expands Platform Support

The latest b9041 release of llama.cpp continues its trend of broadening platform compatibility, making it a versatile choice for developers across different environments. Notably, this update includes support for macOS Apple Silicon with KleidiAI enabled, as well as expanded Vulkan and ROCm 7.2 support on Ubuntu. This release doesn't introduce new models but focuses on enhancing the runtime's adaptability across various hardware configurations. By doing so, llama.cpp strengthens its position as a go-to inference runtime for developers seeking flexibility beyond NVIDIA's CUDA ecosystem.

llama.cpp ReleasesMay 7, 2026

Models & Labsmodels

Llama.cpp Adds Granite-Speech Support

Llama.cpp's latest update expands its functionality by integrating IBM's Granite-Speech, significantly enhancing its audio processing capabilities. The update features a Conformer encoder with Shaw relative position encoding and a QFormer projector, which efficiently compresses audio data into the LLM embedding space. This ensures precise token-for-token matching with HF transformers on audio clips, demonstrating its robustness. By incorporating these advanced audio processing techniques, llama.cpp becomes a more versatile tool for developers, extending its utility beyond text to include sophisticated audio data handling.

llama.cpp ReleasesMay 7, 2026

Models & Labsmodels

llama.cpp b9049 Release Supports MiniCPM-V 4.6

The llama.cpp b9049 release marks a notable step forward by integrating MiniCPM-V 4.6, enhancing the tool's capabilities for developers. This version addresses several bugs and refines features, such as implementing build_attn for flash attention support and improving code style and type checks. The update also extends its reach across various platforms, including macOS, Linux, and Windows, with tailored support for Apple Silicon and Vulkan. These enhancements make llama.cpp a more versatile and reliable tool for developers working with a range of AI models, boosting its performance and usability.

llama.cpp ReleasesMay 7, 2026