AgentLens: Trajectory Evaluation for Code Agents is a benchmark designed to measure what traditional tests ignore. In recent years, the software industry has adopted AI agents capable of writing, reviewing and debugging code autonomously. But most existing evaluations only check whether the agent completes a task in an isolated environment. That simplistic view is misleading in real projects, where the path the agent follows, the mistakes it makes and the way it recovers determine its usefulness in production. AgentLens is not satisfied with the outcome: it examines the entire trajectory.
The proposal combines formal verification with narrative reviews generated by language models. First, it uses objective checks when the task has a verifiable correct answer. Second, it incorporates analysis of the agent's interaction with its tools, its adherence to instructions and its ability to self-evaluate. Finally, it proposes side-by-side comparisons between different versions or models, so each run produces a readable explanation of why that particular score was obtained. This turns evaluation into a source of diagnosis, not just a ranking.
For a software development company like Q2BSTUDIO, this approach is especially valuable. When we build custom software, we cannot allow an AI agent to be a black box. We need to know what decision it made at each step, why it discarded an alternative and how it reacted to an unexpected failure. The traceability offered by AgentLens resembles what we demand from any other critical component: detailed logs, context and auditability. Without that visibility, trusting automation is impossible.
The benchmark also connects with a daily reality in engineering teams: silent regression. An agent can pass all synthetic tests today and fail tomorrow on a similar task because a change in its underlying model altered its behavior. Running AgentLens periodically makes it possible to detect those regressions before they affect users. At Q2BSTUDIO we integrate this logic into our quality pipelines, along with continuous integration practices and deployment on cloud AWS/Azure. The cloud facilitates instrumentation and storage of complete trajectories, turning each run into a valuable information asset for the team.
Cybersecurity also benefits from this methodology. When an AI agent interacts with internal systems, every action represents a risk surface. Trajectory reviews allow identifying anomalous behaviors, unnecessary access or misuse of credentials. AgentLens is not a pentesting tool, but its observability philosophy fits the security requirements of modern organizations. In projects that require continuous auditing, having an interpretable record of what an agent did and why is just as important as the functional tests themselves.
Another area where trajectory evaluation makes a difference is business intelligence. Code agents not only program; they also prepare data, create queries and generate visualizations. A BI/Power BI system can be assisted by an agent that builds reports on demand. The quality of that agent is not measured only by whether the final report conforms to a schema, but by how it explores data, what assumptions it applies and how it communicates findings to the user. The narrative review proposed by AgentLens offers exactly that level of detail.
Side-by-side comparison is one of the most powerful components. Instead of relying on an aggregate metric, teams can present the same task to two agents and compare their strategies. This is very useful during the development of a custom AI agent, when validating a new version before release. The evaluation does not only say which model wins; it explains why it wins. That explanation allows engineers to adjust prompts, tools or decision criteria with a level of precision that was previously unattainable.
The concept of trajectory not only applies to an agent's execution over a few hours. It should also cover the entire software lifecycle: from requirement definition to evolutionary maintenance. An agent working on a legacy codebase needs to understand context, not just generate patches. Evaluation should include the quality of the message it returns to the developer, the clarity of its questions and its ability to ask for help when it lacks enough information. AgentLens captures these behaviors and turns them into actionable data.
Releasing AgentLens as an open source project opens up an enormous experimentation field. Any team can download it, adapt it to its technology stack and contribute new use cases. This is very relevant for the Spanish-speaking software development community, which needs more evaluation tools with clear explanations and not just numbers. The transparency of the benchmark allows decisions about which agent to hire or configure to be made with sound criteria, rather than simply from the result of an automated test.
Another point worth highlighting is the difference between measuring performance and measuring behavior. Classic metrics, such as the percentage of completed tasks or average resolution time, offer a superficial picture. Trajectory evaluation, on the other hand, captures the agent's practical reasoning: how it organizes subtasks, when it decides to ask the user, whether it documents its changes or leaves side effects. These variables are what separate an experimental assistant from a reliable production tool. At Q2BSTUDIO, when we apply this perspective to custom software projects, we get a much more realistic view of the value AI can bring to the business.
The future of code agents depends on trust. And trust is not built with a pass/fail note, but with the ability to inspect every step of the process. AgentLens contributes to this vision by combining formal verification with AI-generated narrative. At a company like Q2BSTUDIO, where we develop custom software, cloud services and artificial intelligence solutions, trajectory evaluation is becoming an internal quality standard. We do not only want to know if something works; we want to know why it works, how it behaves in the face of error and how it can improve.
