Models & Labs
Jev: A Decision-Only AI Model for Agents
Cole Medinhigh confidence
Why it matters
- →Replaces expensive LLM calls for simple routing decisions with ultra-low-cost inference.
- →Offers sub-200ms latency suitable for real-time agent loops and game logic.
- →Introduces a new model class focused purely on probabilistic decision gates.
TypeSafe has launched Jev, an AI model designed exclusively for decision-making rather than text generation. The system accepts state inputs and typed questions, returning decisions with probabilities in approximately 200 milliseconds at a fraction of a cent per call. Early use cases include automating game logic, triaging pull requests, and routing requests between different large language models. This approach positions Jev as a specialized component for agentic coding workflows where speed and cost efficiency are critical.
Read originalMore in Models & Labs
Models & Labsmodels
vLLM v0.30.0: DeepSeek V4.1 and Fast Start
This release is a massive infrastructure overhaul for serving the latest reasoning models. The headline feature is native support for DeepSeek-V4.1-Flash, storing its entire KV cache in MXFP8 on SM100 hardware to drastically reduce memory overhead. For operators tired of slow cold starts, the new Fast Start daemon caches post-quantized weights in GPU memory, allowing engines to map over CUDA IPC instead of reloading from disk. It also brings HiSparse, a host-resident tier that spills KV pages to pinned host memory under pressure, effectively expanding usable context windows without buying more GPUs.
vLLM Releases
Models & Labsother
llama.cpp b11080 release with ROCm 10 and CUDA 13
This update quietly extends llama.cpp’s hardware support to ROCm 10.0 and CUDA 13.4 across Linux and Windows, keeping the library competitive as NVIDIA pushes newer driver stacks. The test suite also gains regex filtering for backend operations, a practical improvement for developers debugging specific inference paths. While no new model architectures are introduced, this release ensures compatibility with the latest GPU ecosystems without forcing users to wait for major version bumps.