StateBench reveals frontier model limits on long-context state tracking

Researchers have released StateBench, a new benchmark designed to test how well language models track entity states across long documents. The study evaluated four frontier models—Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and Luna—and found that while they perform well on simple state updates, accuracy drops sharply on complex tasks involving chains of variable reassignments. To determine if this is an architectural flaw or a training deficit, the team trained smaller models from scratch, revealing that hybrid architectures with recurrent memory (Gated DeltaNet) generalize significantly better than pure transformers on long sequences. The findings suggest that current attention-based models hit a theoretical wall when handling complex state dependencies in extended contexts.