.jpg)
In the evaluation of artificial intelligence agents, success has traditionally been measured by the accuracy of the final response or an aggregated score across the entire trajectory. However, this approach hides critical information: which specific agent action actually drove a positive change in the problem state? Recently, an innovative approach called Agent Step Value (ASV) has emerged, a state transition measurement framework that assigns a score to each observed action based on the change it induces in the expected outcome distribution of a natural language-based evaluator.
ASV relies on state projections before and after each step, using a stateless LLM evaluator that assigns logarithmic scores to fixed candidates. This allows not only detecting when an agent is making constructive or destructive decisions, but also exposing phenomena such as information leakage or floor scores. In a recent study on evidence search tasks in PubMed with a DeepSeek actor, ASV evaluated over a thousand steps and two thousand states, revealing that measures like Bayesian surprise and gold margin gain locate inflection points that final scores and entropy metrics do not capture.
For companies deploying AI agents in critical processes—from automated customer service to legal document analysis—this granularity is essential. It is not enough to know whether the final outcome is correct; one needs to understand which reasoning step or external action (such as a database query or an API call) contributed to success or failure. This is where the ability to develop custom software and custom applications becomes fundamental: only with a flexible platform can a step-by-step evaluation system be integrated that adapts to the particular business logic. At Q2BSTUDIO we offer custom application development that facilitates this integration.
At Q2BSTUDIO we are experts in artificial intelligence for businesses and in creating personalized solutions that allow organizations to get the most out of these advanced techniques. Our services range from implementing AI for businesses to deploying secure infrastructure on AWS and Azure cloud services, as well as cybersecurity and business intelligence services with tools like Power BI. A framework like ASV can be easily integrated into an AI agent ecosystem to provide real-time visibility into the quality of each decision.
Furthermore, the label-free nature of ASV makes it especially attractive for environments where training data is scarce or expensive to obtain. By separating the evaluator's deliberation from the scoring of a single option, candidate probabilities are preserved and anchoring bias is avoided. This fits perfectly with the process automation philosophy we promote: not just automating, but also monitoring and continuously improving.
If your company is exploring the use of intelligent agents for complex tasks—such as information retrieval, diagnosis, or recommendations—we recommend considering the implementation of step metrics like ASV. To do this, having a technology partner that offers custom software and experience in the full development cycle is key. At Q2BSTUDIO we can help you design and deploy advanced evaluation systems that truly add value to the business.
