Agents
Benchmarking Open Models for Agentic Use
Hugging Face Bloghigh confidence
Why it matters
- →Provides a new way to evaluate software libraries for agentic use.
- →Aims to improve efficiency and reduce costs in agent-driven tasks.
- →Encourages better design of APIs and documentation for machine interaction.
Hugging Face has developed a new benchmarking tool to evaluate the efficiency of coding agents interacting with software libraries, specifically focusing on transformers. This tool measures not only the accuracy of the agents' outputs but also the process efficiency, including steps and resources used. The goal is to optimize libraries for agentic use, ensuring APIs and documentation are accessible and efficient for autonomous agents. This could lead to significant improvements in how agents perform tasks, potentially reducing costs and enhancing performance.
Read originalMore from Hugging Face Blog
© Hugging Face BlogResearchresearch
MosaicLeaks Tackles Privacy in AI Research Agents
MosaicLeaks introduces a critical challenge for AI research agents by addressing the privacy risks inherent in their web queries. The research reveals how agents can unintentionally disclose sensitive information through seemingly harmless queries, a situation termed the mosaic effect. To mitigate this, the team developed Privacy-Aware Deep Research (PA-DR), a training method that significantly reduces information leakage from 34% to 9.9% while preserving task performance. This innovative approach enables agents to conduct more web searches without compromising privacy, marking a significant advancement in balancing AI functionality with data protection.
Hugging Face Blog
© Hugging Face BlogResearchresearch
Exploring Alternatives to LoRA in Fine-Tuning
Hugging Face's latest exploration into parameter-efficient fine-tuning (PEFT) techniques challenges the dominance of LoRA, a popular method for reducing memory requirements in model fine-tuning. While LoRA is widely used due to its early adoption and extensive support, the PEFT library now offers a comprehensive benchmarking framework to objectively evaluate various techniques. This initiative reveals that other methods can outperform LoRA in specific scenarios, suggesting that users might benefit from considering alternatives based on their unique needs. The findings encourage a more nuanced approach to model fine-tuning, potentially leading to better performance and efficiency.
Hugging Face Blog
More in Agents
© Cole MedinAgentsagents
AI Engineers Use Loops Instead of Prompts
AI engineers are moving away from manually prompting their agents, opting instead for automated loops that handle the task. This approach, highlighted by the head of Claude Code at Anthropic, involves using orchestrators to manage tasks and spin up agents in parallel, reducing the need for direct human intervention. However, this method can be costly and prone to compounding errors, necessitating robust systems with observability. A new TypeScript app has been developed to address these challenges, running agents in a loop and providing a dashboard for monitoring. This shift represents a significant change in how AI systems are managed, offering more control and efficiency.
Cole Medin
© WIRED AIAgentsagents
IO-AI Tech Innovates Robot Teleoperation in Shenzhen
IO-AI Tech is pioneering a new frontier in robotics by enabling workers to control humanoid robots using VR headsets and motion-tracking gear. This approach allows robots to perform tasks like stocking shelves and picking items, while also collecting valuable training data for future autonomous operations. The startup's technology is particularly significant in Shenzhen, a hub for manufacturing, where it collaborates with local companies to integrate robots into production lines. This development could accelerate the deployment of AI-powered automation in various industries, offering a glimpse into the future of blue-collar work.
WIRED AI
© The Verge AIAgentsagents
Genesis AI unveils non-humanlike robot Eno
Genesis AI is challenging traditional notions of humanoid robots with its new creation, Eno. Unlike typical humanoid robots, Eno is designed around human capabilities rather than appearance, featuring human-like hands for tool use but lacking a human-like form. This approach allows Eno to function as a general-purpose robot, adaptable to various tasks across industries. With plans to begin production by 2026, Genesis AI aims to deploy Eno in sectors like manufacturing and logistics, eventually expanding to consumer markets. This marks a shift in how robots are designed to interact with human environments.
The Verge AI