Benchmark report

Task verified

9m 19s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The cluster-size SKU query parameter mismatch (kind=postgresql silently returned MySQL sizes) caused a 422 on the first database create and an extra lookup round-trip.

Where the time went

9:20 agent time
  • Research0:32
  • Attempt7:07
  • Verify1:41
9m 19sTotal elapsed
5:05Time to first success
0:00Waited on the owner

Judge reason

Live PlanetScale management API evidence supports every criterion. Database tpcbench-e83b39c8 created (POST /organizations/rick-promptingco/databases, kind=postgresql, HTTP 201, 03_create_database.json). Admin role (inherited_roles:[postgres]) created table items and seeded 2 rows over psql with PGSSLMODE=verify-full, PGSSLROOTCERT=system (08). Scoped credential created via the documented Roles API POST .../branches/main/roles with inherited_roles:[pg_read_all_data] (HTTP 200, 09) — docs cited (create_role.md, postgres/connecting/roles.md). Connecting with only that role: current_user=pscale_api_yb8sty9rz5ab, SELECT returned 2 rows, INSERT refused with 'ERROR: permission denied for table items' (psql exit 1), pg_auth_members shows pg_read_all_data (10, matching transcript L111 live output). TLS proven by \conninfo 'SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384)' plus a bad-CA negative control that fails (11). Revocation: DELETE role -> HTTP 204, follow-up GET -> 404, absent from list_roles, reconnect -> 'FATAL: password authentication failed' (12-15). Cleanup: admin role deleted (204/404/empty list, reconnect fails), database DELETE -> 204, GET -> 404, absent from list_databases (16-22). Leak scan over 67 files for the token and both role passwords found none; all saved passwords are <redacted>, passwords passed only via child PGPASSWORD. No mocks or hard-coded responses.

Attempt record

  1. 1:29Answered from saved key: PLANETSCALE_SERVICE_TOKENno wait
  2. —First success verified

Attempt artifacts saved.