Benchmark report

Task verified

10m 9s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The agent's own no-TCP guard rejected the sandbox's mandatory HTTP CONNECT proxy hop, forcing two failed runs and two patches before the query executed.

Where the time went

10:09 agent time
  • Research0:25
  • Attempt7:26
  • Verify2:18
10m 9sTotal elapsed
5:25Time to first success
0:00Waited on the owner

Judge reason

Verified end to end from saved artifacts. Database tpcbench-03812437d8e3 (kind postgresql) created via POST api.planetscale.com/v1 with Bearer service token (04-create-database.json, HTTP 201), polled to state ready (05, 200), Postgres role created on branch main (07-create-role.redacted.json, password [REDACTED]). work/code/query-over-http.mjs uses @neondatabase/serverless 1.2.0 neon() with neonConfig.fetchEndpoint = (host) => https://${host}/sql — no Pool/Client, WebSocket constructor replaced with a thrower, and net/tls connect wrapped to hard-fail on anything but :443 or the sandbox CONNECT proxy. Runtime output (transcript idx 146 and 08-http-query-stdout.txt / 08-http-query-results.json) shows 4 POSTs to https://us-east-4.pg.psdb.cloud/sql all status 200: CREATE TABLE items, two-row INSERT ... RETURNING, SELECT returning id=1 widget 7 and id=2 gadget 42 (matching the saved rows), plus version() = PostgreSQL 18.6; sockets_opened lists only proxy:8080 hops and tls.connect to :443, violations empty. Cleanup graded: DELETE role 204 after documented successor=postgres retry of the 422 (09), GET role 404 (10), DELETE database 204 (11), GET database 404 (12), and list of all org databases contains zero tpcbench matches (13, grep count 0). Grep over /workspace/home finds no password or password-bearing connection string — only the shell template with ${P} in run-query.sh. Nothing mocked or hard-coded.

Attempt record

  1. 1:19Answered from saved key: PLANETSCALE_SERVICE_TOKENno wait
  2. —First success verified

Attempt artifacts saved.