Models & Labs
Jun Kim joins Hugging Face to lead MLX ecosystem
Hugging Face Bloghigh confidence
Why it matters
- →Professionalizes maintenance for Apple Silicon's primary local AI framework.
- →Reduces friction in porting Hugging Face models to MLX.
- →Strengthens the non-NVIDIA AI ecosystem through strategic hiring.
Hugging Face has hired Jun Kim, the creator and maintainer of oMLX, to lead efforts supporting the MLX community on Apple Silicon. Kim will transition from maintaining oMLX as a side project to a fully funded role at Hugging Face, ensuring long-term stability for the framework. The collaboration aims to streamline the process of converting models from the Hugging Face Transformers library into MLX implementations. This partnership seeks to strengthen ties between key projects like mlx-lm and LMStudio, providing better tools for local AI inference on Mac hardware.
Read originalMore from Hugging Face Blog
© Hugging Face BlogResearchmodels
Physics-based pruning beats heuristics for LLM compression
Multiverse AI reframes block removal as an Ising glass optimization problem, capturing the hidden couplings between transformer layers that mean-field methods ignore. By mapping block importance to spin interactions via a Hessian matrix, they turn model compression into a search for low-energy states rather than independent block scoring. This approach yields a massive 23-point MMLU gain over existing baselines when compressing Llama-3.3-70B by half, proving that many-body physics tools can unlock deep compression without retraining. The method scales to large models using classical and quantum-inspired solvers, offering a rigorous alternative to heuristic pruning.
Hugging Face Blog
© Hugging Face BlogCoding Toolscoding
Hugging Face tokenizers v1 delivers massive speed gains
Hugging Face has rewritten its core tokenization library to eliminate the CPU bottleneck that often starves GPUs during inference. By replacing generic regex engines with hand-written SIMD splitters and eliminating memory allocations in the merge loop, version 1 encodes text up to 30 times faster than v0.23 on Apple Silicon. The rewrite preserves exact output compatibility while scaling efficiently across multiple threads. This shift ensures data pipelines no longer idle waiting for tokenization, a critical optimization for high-throughput serving environments.
Hugging Face Blog
More in Models & Labs
Models & Labsmodels
vLLM v0.30.0: DeepSeek V4.1 and Fast Start
This release is a massive infrastructure overhaul for serving the latest reasoning models. The headline feature is native support for DeepSeek-V4.1-Flash, storing its entire KV cache in MXFP8 on SM100 hardware to drastically reduce memory overhead. For operators tired of slow cold starts, the new Fast Start daemon caches post-quantized weights in GPU memory, allowing engines to map over CUDA IPC instead of reloading from disk. It also brings HiSparse, a host-resident tier that spills KV pages to pinned host memory under pressure, effectively expanding usable context windows without buying more GPUs.
vLLM Releases
Models & Labsother
llama.cpp b11080 release with ROCm 10 and CUDA 13
This update quietly extends llama.cpp’s hardware support to ROCm 10.0 and CUDA 13.4 across Linux and Windows, keeping the library competitive as NVIDIA pushes newer driver stacks. The test suite also gains regex filtering for backend operations, a practical improvement for developers debugging specific inference paths. While no new model architectures are introduced, this release ensures compatibility with the latest GPU ecosystems without forcing users to wait for major version bumps.
llama.cpp Releases
Models & Labsother
llama.cpp b11081 release with CUDA 13 and ROCm 10 support
This release quietly cements llama.cpp as the universal inference runtime by adding default builds for CUDA 13.4 and ROCm 10.0, effectively closing the gap on newer NVIDIA and AMD hardware without requiring manual compilation flags. The inclusion of KleidiAI for Apple Silicon remains a key differentiator for local Mac users seeking optimized ARM kernels. While the changelog details internal test improvements like configurable tensor standard deviation, the real value lies in the expanded binary matrix that supports developers across the latest GPU architectures. Readers can now deploy on cutting-edge hardware out of the box. The update ensures that teams using the newest graphics cards do not need to wait for source code patches to achieve stable performance. This is a practical step toward making local inference truly hardware-agnostic.
llama.cpp Releases