Models & Labs
LFM2.5-DSpark Boosts Inference Speed by Up to 3.2x
Hugging Face Bloghigh confidence
Why it matters
- →LFM2.5-DSpark significantly reduces inference latency, enhancing AI model responsiveness.
- →The integration with llama.cpp and SGLang ensures broad compatibility and ease of use.
- →This development makes AI applications more efficient on both high-end and consumer hardware.
Hugging Face has introduced LFM2.5-DSpark, a new approach that accelerates AI model inference by up to 3.2 times. This improvement is achieved through speculative decoding, which uses draft models to propose tokens that are verified in a single pass by the target model. The integration supports llama.cpp and SGLang, allowing for immediate deployment on various platforms. This advancement enhances performance on both high-end GPUs and consumer devices, significantly reducing latency and improving user interactivity.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
More from Hugging Face Blog
© Hugging Face BlogGeneral AIother
Ai2 replaces priority scheduling with GPU time budgets
Allen Institute for AI solved the 'tragedy of the commons' in its H100 and B200 clusters by abandoning priority queues for a budget-based system. Researchers now spend allocated GPU time rather than hoarding it, turning resource allocation into a transparent administrative process. This shift eliminates squatting and priority inflation while keeping occupancy high through hierarchical fair-share scheduling. It proves that treating compute as a financial asset works better than treating it as a shared utility.
Hugging Face Blog
© Hugging Face BlogMore in Models & Labs
Models & Labsmodels
llama.cpp b11535 release with ROCm 10 and CUDA 13
This release quietly cements llama.cpp as the universal inference runtime by adding default support for ROCm 10.0 and CUDA 13.4 across Linux and Windows. AMD GPU users finally get parity with NVIDIA's latest driver stack without manual configuration, while Apple Silicon KleidiAI builds are temporarily disabled to resolve stability issues. The inclusion of Snapdragon NPU support on Linux signals a serious push into edge AI hardware beyond just x86 and ARM CPUs. It is less about new features and more about ensuring the toolchain keeps pace with the rapidly evolving GPU landscape.
llama.cpp Releases
Models & Labscoding
llama.cpp b11540 adds ROCm 10 and CUDA 13 support
This release quietly closes the hardware gap for local inference by adding default builds for ROCm 10.0 and CUDA 13.4 across Linux and Windows. AMD users finally get parity with NVIDIA in the binary distribution, while CUDA 13 support future-proofs setups on newer drivers. The inclusion of Snapdragon and OpenVINO binaries further broadens the hardware surface area without requiring custom compilation. It is a pragmatic update that makes llama.cpp the most accessible runtime for diverse local AI hardware.
