Models & Labs
Together AI introduces Polymorphic Inference and Compiler
Together AI Bloghigh confidence
Why it matters
- →Shifts inference optimization from manual kernel writing to automated agentic search.
- →Enables faster adaptation to new open-source model architectures and traffic patterns.
- →Reduces operational overhead for serving diverse OSS models at scale.
Together AI has announced 'Polymorphic Inference,' a strategic shift from static optimization to dynamic, agent-driven performance tuning for open-source models. Central to this approach is Together Compiler, an agentic system that automatically identifies and applies optimization patterns across varying workloads and hardware. The company reports the compiler is already active in production on serverless endpoints, aiming to reduce the latency typically associated with new model deployments. This move positions Together AI to handle the increasing diversity of open-source architectures without relying on manual kernel engineering.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
How we got here
- 1ZML Launches Free Inference Software for AI Chips
TechCrunch AI · July 8, 2026 · Related
- 2Intel Explores Agentic AI for Enterprise Workflows
MIT Technology Review AI · July 27, 2026 · Related
- 3Together AI Enhances Model Inference Configuration
Together AI Blog · July 29, 2026 · Same story
- 4Together AI Introduces Autoscaling for LLM Inference
Together AI Blog · July 31, 2026 · Same story
- 5Hugging Face Introduces ALTK-Evolve for Efficient AI Agents
Hugging Face Blog · August 11, 2026 · Related
- 6Claude Code relaunches Projects for multi-agent coordination
The Verge AI · September 17, 2026 · Related
- 7Jev: A Decision-Only AI Model for Agents
Cole Medin · September 21, 2026 · Related
- 8Liquid AI releases efficient on-device agent model
Hugging Face Blog · October 13, 2026 · Related
Follow this story
Open the full story →Together AI automates inference optimization with agents
2 developments
- Nov 4 · Together AI Blog
Together AI automates inference optimization with agents - Nov 4 · Together AI Blog
Together AI introduces Polymorphic Inference and Compiler (This article)
More in Models & Labs
Models & Labsother
llama.cpp b12177 adds CUDA 13 and ROCm 10
This release quietly closes the hardware gap for local inference by adding default support for CUDA 13.5 and ROCm 10.0 alongside existing CUDA 12 builds. AMD GPU users finally get first-class parity with NVIDIA, while Snapdragon mobile chips gain dedicated NPU acceleration paths. Apple Silicon KleidiAI is explicitly disabled here, a notable regression for Mac performance that suggests ongoing tuning issues. The inclusion of OpenVINO and SYCL backends further cements llama.cpp as the universal runtime for inference across nearly every major silicon architecture.
llama.cpp Releases
Models & Labsmodels
llama.cpp adds Kimi K3.1-Pro support
Kimi’s new K3.1-Pro model joins the llama.cpp ecosystem, bringing a distinct architectural twist to local inference. By replacing full-attention MLA layers with DSA (Dynamic Sparse Attention) and linear-attention KDA layers, this release tests how well sparse caching handles hybrid attention patterns. The implementation keeps indexer precision high to avoid quality loss during top-k selection, ensuring the model runs accurately on consumer hardware. This expands the library’s reach into newer Chinese open-weight models that rely on complex memory management.
llama.cpp Releases
Models & Labsother
llama.cpp b12182 release with CUDA 13 and ROCm 10
This release quietly cements llama.cpp as the universal inference runtime by adding default builds for CUDA 13.5 and ROCm 10.0 across Linux and Windows. The inclusion of ROCm 10 means AMD GPU users no longer need custom compilation to access modern acceleration, effectively closing the gap with NVIDIA's ecosystem support. While KleidiAI on Apple Silicon is disabled in this build, the broader platform coverage ensures that local inference remains accessible on virtually any hardware configuration without manual setup.
llama.cpp Releases