Models & Labs
NVIDIA Dory: 4B Looped Transformer
Sam Witteveenhigh confidence
Why it matters
- →Demonstrates that architectural innovation can outperform simple parameter scaling for edge deployment.
- →Offers a viable path for running high-quality reasoning models on hardware with limited VRAM.
- →Challenges the prevailing assumption that larger models are always necessary for complex tasks.
NVIDIA has released Dory, a 4-billion-parameter language model utilizing a looped transformer architecture, available via the Nemotron Labs collection on Hugging Face. Unlike standard transformers that process tokens sequentially with distinct weights, Dory reuses its weight matrix across multiple 'loops' per token, allowing it to simulate deeper processing without increasing parameter count. Benchmarks indicate it competes with significantly larger models like Gemma 2 27B in specific reasoning tasks while offering superior inference speed and lower memory usage. This approach highlights a growing industry focus on architectural efficiency as an alternative to scaling model size.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
How we got here
- 1NVIDIA Unveils Nemotron 3.5 Lightning and NeMo Switchyard
NVIDIA Blog · August 11, 2026 · Related
- 2NVIDIA Boosts Local AI with New Open Models
NVIDIA Blog · August 11, 2026 · Related
- 3Nvidia Highlights Importance of AI Harness Over Model
TechCrunch AI · August 21, 2026 · Related
- 4NVIDIA Jetson Orin Nano 2 targets edge AI for robots
AI News · August 26, 2026 · Related
- 5llama.cpp optimizes NVIDIA V100 inference performance
llama.cpp Releases · October 2, 2026 · Related
- 6Nemotron Fine-Tuning Hits Gold at IOI and IMO
Hugging Face Blog · October 7, 2026 · Related
- 7NVIDIA and AI4Bharat release 2.8T Indic tokens
Hugging Face Blog · October 16, 2026 · Related
- 8Nvidia releases open Nemotron models for agents and PCs
TechCrunch AI · October 19, 2026 · Related
What happened next
- 9llama.cpp b12064 optimizes Hexagon NPU permutes
llama.cpp Releases · November 3, 2026 · Related
More from Sam Witteveen
More in Models & Labs
Models & Labscoding
vLLM v0.33.0: Blackwell & DeepSeek V4.1 Optimizations
vLLM v0.33.0 is a massive infrastructure update that finally brings first-class support for NVIDIA's Blackwell architecture and the new DeepSeek V4.1 models. The release drops CUDA 12.9 in favor of CUDA 13.0, signaling a hard pivot to next-gen hardware while adding critical optimizations like block-diffusion drafters and quantized LM heads that significantly boost decode throughput. AMD users also get major wins with ROCm 10.0 defaults and MonoKernel decoding on MI355X. This isn't just another version bump; it's the runtime catching up to the bleeding edge of model complexity and silicon capability.
vLLM Releases
Models & Labsother
llama.cpp b12054 adds CUDA 13 and ROCm 10.0
This release quietly cements llama.cpp as the universal inference runtime by adding default support for CUDA 13 and ROCm 10.0. NVIDIA users can now leverage newer driver stacks without waiting for separate builds, while AMD GPU owners finally get parity with modern ROCm versions. The inclusion of Snapdragon and OpenVINO binaries further widens the hardware net, though Apple Silicon KleidiAI is notably disabled here. It’s less about new features and more about ensuring no major accelerator is left behind in the local LLM ecosystem.
llama.cpp Releases
Models & Labsmodels
llama.cpp adds Clover MoE model support
llama.cpp quietly expands its architecture roster by adding native support for the Clover Mixture of Experts model. This isn't just a wrapper; it implements causal hybrid attention and reuses existing recurrent-state and LoRA helpers, meaning local inference for this specific architecture is now viable on consumer hardware. The release includes GGUF conversion tools and loads across all major backends including CUDA 13, ROCm 10, and Apple Silicon. For researchers tracking non-transformer or hybrid architectures, this closes a gap in the most popular local inference runtime.
llama.cpp Releases