Benchmark report

Task verified

6m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The stale documented revoke endpoint .../auth/tokens/invalidate 404'd, forcing a fetch of turso.tech/openapi.json to find auth/rotate; everything else ran first try.

Where the time went

6:25 agent time
  • Research0:25
  • Attempt4:19
  • Verify1:41
6m 25sTotal elapsed
1:35Time to first success
0:00Waited on the owner

Judge reason

Saved code (out/run_task.sh, api.sh, sqlrun.py) and live responses show the full lifecycle against api.turso.tech: DB tpcbench-38819420c9f4 created (01-create-database.json, HTTP 200); table items created and seeded over https .../v2/pipeline (03, TLSv1.3); scoped credential created via the documented management API POST .../auth/tokens?authorization=read-only (openapi createDatabaseToken excerpt in 00), claims a=ro (04b); with only that token SELECT returned rows alpha/beta and INSERT was refused with code BLOCKED 'SQL write operations are forbidden' (05); revoke via POST .../auth/rotate (operationId invalidateDatabaseTokens) HTTP 200 (06), after which the token got HTTP 401 'role was invalidated after token was issued' (07, repeated 6x in transcript); DELETE returned {"database":"tpcbench-38819420c9f4"} 200 and the follow-up GET 404 'record not found' plus an org list with 0 tpcbench- databases (08, 09). TLS satisfied by the https libSQL URL with system-CA verification. No JWT, password, or password-bearing connection string appears in any saved file (grep for eyJ-prefixed strings found none; create-database returned password:""). Turso has no SQL users, so the credential identity is shown by the JWT claims — an accurate reflection of the product, not a substitution. The only unused resource, group 'default', pre-existed the run.

Attempt record

  1. 0:53Answered from saved key: TURSO_API_TOKENno wait
  2. —First success verified

Attempt artifacts saved.