Models & Labs
Together AI Enables A/B Testing for LLMs
Together AI Bloghigh confidence
Why it matters
- →Enables precise measurement of model performance with real user data.
- →Simplifies traffic routing and experiment management at the endpoint level.
- →Facilitates data-driven decisions for optimizing AI model deployments.
Together AI has launched a new A/B testing feature for large language models in production environments. This tool allows developers to split live traffic between a control model and up to 20 variants, enabling real-world performance comparisons based on user engagement metrics. The platform handles traffic routing at the endpoint level, simplifying the process and avoiding complex client-side configurations. This advancement provides a streamlined approach for teams to assess and improve AI models based on actual user interactions, ensuring more effective deployment decisions.
Read originalMore in Models & Labs
Models & Labsmodels
Llama.cpp b10456 Release Boosts Performance
The latest b10456 release of llama.cpp brings a significant performance boost, particularly in the quantized copy kernel launches. By adjusting the thread and block count to better match the size of the quant, the update notably enhances throughput on the Arc 70 from 20.21 GB/s to 158.19 GB/s in the q4_0 to f32 path. While other quant paths remain unchanged in performance, this improvement marks a substantial leap for specific use cases. This release continues to refine llama.cpp's capabilities, making it more efficient for developers working with quantized models.
llama.cpp Releases
© Hugging Face BlogModels & Labsresearch