New ask Hacker News story: Ask HN: What metrics do you track when building agents?

Ask HN: What metrics do you track when building agents?
2 by rstagi | 0 comments on Hacker News.
The main metric I look at is inference cost, which is a pretty good indicator when something's off. However, a couple of days a go I discovered we were loading ~300 tools due to a bug, affecting both costs and accuracy. It was going on for some time and I didn't realize it by looking at the costs, cause it got hidden by other changes I made in the same period. Now I started tracking that and some other metrics, but it made me wonder what else I might be missing. So: what are you tracking? Any tips? Curious to hear

Comments

Post a Comment

Popular posts from this blog

New ask Hacker News story: EVM-UI – visual tool to interact with EVM-based smart contracts

EVM-UI – visual tool to interact with EVM-based smart contracts 21 by magnusgraviti | 8 comments on Hacker News. Hey HN! We've built https://evm-ui.com, a free tool that allows you to interact with smart contracts visually. It started as an internal project while building Web3 apps for clients — we needed something more flexible and lightweight than Remix or Etherscan's UI. Unlike Etherscan or Polygonscan, you don't need to verify and open-source your smart contract to get a working UI. Paste the ABI (or load it from a template) and proceed. With EVM-UI, you can - Import contracts by address + ABI, or paste ABIs directly - Read/write functions with auto-generated UI (supports all Solidity types, including arrays) - Execute actions with smart inputs (dropdowns, min/max helpers, etc.) - Switch between EVM chains, testnets, or local RPCs within one workspace - Use developer tools (Keccak256, ABI encoder/decoder, address utils) - Share environments with your team and your cl...
Read more

How can Utilize Call Center Outsourcing for Increase your Business Income well?

Image
Call center outsourcing is not uncommon however, not every company can live up to the expectations of a great client. The service provider should have the ability to assist you obtain access to new abilities and competences in order to help you achieve efficiency and effectiveness in your company. Digital office providers assist business in providing unique, innovative solutions that are intended to custom-fit your personal small business needs. Some back office service providers also have fully-developed outbound calling and telemarketing services that you can utilize. The world has taken a huge leap with the technological revolution. What was impossible before now seems to be a matter of seconds. As we see more and more start-ups and digital companies emerging, we need to understand how we can account the rising number of BPO companies that can be helpful to us. Running a call center internally may be an expensive affair particularly when it...
Read more