Coding Tools
Hugging Face tokenizers v1 delivers massive speed gains
Hugging Face Bloghigh confidence
Why it matters
- →Eliminates CPU-bound bottlenecks in high-throughput inference pipelines.
- →Achieves up to 30x speedup without changing token IDs or API contracts.
- →Reduces memory allocation overhead via scratch buffers and thread-local caching.
Hugging Face has released the release candidate for tokenizers v1, a complete rewrite of its Python and Rust library focused on performance. The update claims encoding speeds up to 30 times faster than version 0.23 by replacing regex-based pre-tokenization with SIMD-optimized hand-written splitters and removing memory allocations from the BPE merge loop. Benchmarks show 76% linear scaling across eight workers on Apple M4 Max hardware, with identical token ID outputs to ensure backward compatibility. The library now supports broader model families and aims to integrate these optimizations directly into the transformers library upon final release.
Read originalMore from Hugging Face Blog
© Hugging Face BlogModels & Labsother
Jun Kim joins Hugging Face to lead MLX ecosystem
Hugging Face is professionalizing the MLX ecosystem by hiring Jun Kim, creator of oMLX, to work full-time on Apple Silicon AI. This move signals a serious commitment to stabilizing local inference outside NVIDIA's CUDA monopoly. By integrating oMLX into Hugging Face’s infrastructure, they aim to streamline the conversion of Hugging Face models into MLX-compatible formats. It removes the bottleneck of volunteer maintenance for a framework that is becoming essential for Mac-based AI development.
Hugging Face Blog
© Hugging Face BlogResearchmodels
Physics-based pruning beats heuristics for LLM compression
Multiverse AI reframes block removal as an Ising glass optimization problem, capturing the hidden couplings between transformer layers that mean-field methods ignore. By mapping block importance to spin interactions via a Hessian matrix, they turn model compression into a search for low-energy states rather than independent block scoring. This approach yields a massive 23-point MMLU gain over existing baselines when compressing Llama-3.3-70B by half, proving that many-body physics tools can unlock deep compression without retraining. The method scales to large models using classical and quantum-inspired solvers, offering a rigorous alternative to heuristic pruning.
Hugging Face Blog
More in Coding Tools
Coding Toolscoding
llama.cpp adds env var control for server args
This release solves a real operational headache: configuring llama-server parameters like temperature and penalties via environment variables instead of command-line flags. By supporting LLAMA_ARG_* prefixes, it enables clean integration with systemd EnvironmentFile directives, making deployment on Linux servers significantly more robust and reproducible. The update also refreshes documentation using llama-gen-docs to keep the readme accurate. While no new inference engines are added, this is a practical step toward treating local LLM serving as standard infrastructure rather than a manual script.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11090 fixes CUDA Volta build
This release quietly repairs a regression in the CUDA backend that broke compilation on older Volta GPUs. By generalizing the tile shape logic in the load_ldmatrix kernel, the maintainers restored support for devices like the Tesla V100 without sacrificing newer architectures. It is a necessary patch for anyone running inference on legacy hardware, ensuring the codebase remains usable across the full spectrum of NVIDIA cards rather than drifting toward only the latest silicon.
llama.cpp Releases
Coding Toolscoding
llama.cpp b11093 adds CUDA 13 and ROCm 10.0
This release quietly closes the hardware gap for local inference by adding default support for CUDA 13 and ROCm 10.0 alongside existing CUDA 12 builds. Users with newer NVIDIA GPUs or AMD accelerators no longer need to compile from source to get hardware acceleration, effectively making llama.cpp the universal runtime for consumer and enterprise AI chips. The simultaneous availability of both major driver stacks means developers can target the latest hardware without maintaining separate build pipelines. This standardization removes a significant friction point for anyone deploying models on modern, non-CUDA-12-only infrastructure.
llama.cpp Releases