Benchmark report

Task verified

3m 16s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The PEP 668 externally-managed Python environment rejected the plain pip install, costing one extra step to create a venv; nothing product-side slowed the agent.

Where the time went

3:16 agent time
  • Starting0:01
  • Research0:15
  • Attempt1:47
  • Verify1:15
3m 16sTotal elapsed
1:24Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (agentmail-task/create_delete_inbox.py) makes real HTTPS calls to https://api.agentmail.to with a bearer key read from a private file (no mocks, no hard-coded responses). evidence/run_output.txt and evidence/api_transcript.json show the full live sequence with server Date headers: POST /v0/inboxes -> 200 returning inbox_id/email nastyguitar459@agentmail.to (both printed by the script), GET /v0/inboxes/{id} -> 200 confirming it existed, DELETE /v0/inboxes/{id} -> 202 Accepted empty body, then bounded polling inside the 60s deadline: poll 1 -> 409 resource_deleting (17:57:05), poll 2 -> 404 not_found (17:57:07). The report states the documented end state (inbox deleted, GET returns 404 not_found) rather than the intermediate 202/409. Setup: docs fetched from auth.md plus the create/delete API reference, key obtained via benchmark-request, agentmail SDK + requests installed in a venv after a PEP 668 externally-managed-environment failure.

Attempt record

  1. 0:47Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.