Benchmark report

Task verified

5m 21s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Little slowed it: one self-inflicted jq quoting error (idx 23) and the post-delete GET returning 403 instead of 404, which needed a list-projects read to confirm removal.

Where the time went

5:22 agent time
  • Research0:23
  • Attempt3:42
  • Verify1:16
5m 21sTotal elapsed
2:23Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/01-05*.sh) drives the real Management API at https://api.supabase.com/v1 with curl. Transcript idx 56 shows POST /v1/projects -> HTTP 201 creating project ref hmeuyjgwushjryehrsfr, name tpcbench-5a415d8da2ff97a7, region us-east-1, status ACTIVE_HEALTHY (artifacts/02,03). Pooler details came from GET /v1/projects/{ref}/config/database/pooler (artifacts/04: aws-0-us-east-1.pooler.supabase.com:6543, user postgres.hmeuyjgwushjryehrsfr). Stock psql 16.15 connected with sslmode=require (artifacts/05 params) and returned 'select_1 = 1' plus PostgreSQL 17.11 server version (artifacts/06, transcript idx 74); artifacts/08 shows \conninfo 'SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384)' and sslmode=disable refused, with pg_stat_ssl ssl=f as explicitly permitted behind Supavisor. DELETE /v1/projects/{ref} returned HTTP 200 with the ref echoed (artifacts/09, idx 96), and follow-up reads confirm removal: GET /v1/projects/{ref} -> 403 no-longer-accessible (artifacts/10) and GET /v1/projects lists only the pre-existing 'Rick sandbox' ref (artifacts/11). No password or password-bearing URL appears in any saved file: the pooler template keeps [YOUR-PASSWORD], saved params show [REDACTED], and a leak scan over all files reported 'LEAKS: none' (idx 107). Nothing was mocked or hard-coded.

Attempt record

  1. 1:15Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.