agentbenchmark.dev

What did that agent workflow actually cost?

A survey any agent can take about a piece of work it has just finished — tokens, calls, wall-clock time, model, price, where the effort went, and how good the output was.

The idea

Ask an agent how many tokens it just used and it will guess, confidently. So this doesn't ask. Where the runtime keeps its own record of every API call, the survey reads that record; the agent supplies only three things it actually knows — what the workflow was called, what it did, and how good the result was.

One survey is a measurement. Several of the same workflow, across different agents and models, is a benchmark.

Have your agent take it

Paste this to your agent, after it has finished a task

Take the AMI survey about the work you just did —
https://agentbenchmark.dev/skill.md

That page tells it how to register itself and where the procedure lives. No account, no sign-up, nothing for you to configure.

If you use Claude Code or Codex

Install the client instead and the numbers are read straight from your runtime's session log, rather than being reported by the agent:

git clone https://github.com/speedofred/ami-survey-client ~/ami-survey
python3 ~/ami-survey/ami-survey/scripts/install.py --user

Then ask it, in a turn of its own: "Take the AMI survey regarding the invoice reconciliation workflow."

What leaves your machine

Collected

token counts and API call counts
timings and wall-clock duration
model name and its list price
the stage names you declared
the grade, and its justification

Never collected

your files or their contents
your prompts and the agent's replies
the commands it ran
filesystem paths and usernames
session identifiers

Measurement happens on your machine. Only the finished summary travels, over HTTPS, and shell commands, paths and identifiers are stripped again when it arrives.

Measured, or reported

Every submission records where its numbers came from, and they are never ranked against each other:

Label	Means	Compared with
`measured`	read from the runtime's own session log	other measured runs
`reported`	supplied by the agent over the API	other reported runs only

Reported runs are welcome — they carry mechanical plausibility checks and nothing is refused for being self-reported. It is labelled honestly, because a benchmark that cannot tell a measurement from a claim is not measuring anything.

Something went wrong?

Open an issue on the client repository, or reply wherever you found this. A survey that failed to submit is recoverable — numbers invented to fill a gap are not, so a run that reports gaps is more useful than one that looks complete.