Benchmark report

Task verified

8m 54s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current AAInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Driver install friction (externally-managed pip, then PYTHONPATH under python3 -I) plus ~2.5 min database provisioning poll and three rejected role-delete attempts.

Where the time went

8:55 agent time
  • Research0:45
  • Attempt6:45
  • Verify1:24
8m 54sTotal elapsed
4:50Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. work/out/04-create-database.json shows POST /organizations/rick-promptingco/databases -> 201 for tpcbench-4dae81fd (kind=postgresql), 05 shows state=ready, 07-create-role.json shows a 200 role with password redacted. code/run_sql.py connects via psycopg 3.3.8/libpq 18.0.6 with sslmode=verify-full and sslrootcert=system (recorded in 08-sql-run.json connection_parameters, host us-east-3.pg.psdb.cloud) and reports PostgreSQL 18.6, proving a real TLS connect. It issues CREATE TABLE items (id integer PRIMARY KEY, name text NOT NULL), three INSERT INTO items (id,name) VALUES (%s,%s) with bound (1,alpha)/(2,beta)/(3,gamma) (status INSERT 0 1 each) and SELECT id,name FROM items WHERE id >= %s ORDER BY id with bound 2 -- no string concatenation. Transcript line 86/87 shows the live run printing exactly '(2, 'beta')' then '(3, 'gamma')', matching 08-sql-stdout.txt and the rows in 08-sql-run.json. Cleanup: 10-delete-database.json DELETE -> 204, 11 GET -> 404 not_found, 12 list shows zero tpcbench-* databases. Role-only delete returned 422 three times (09/09b/09c) but the database delete removed it. No password or password-bearing URL appears in any saved file (verified by grep; 07 and 08 carry <REDACTED>); the agent's own scan matched only the literal format template in build_url.py. pg_stat_ssl ssl=f is the documented pooler artifact and is optional.

Attempt record

  1. 1:28Answered from saved key: PLANETSCALE_SERVICE_TOKENno wait
  2. —First success verified

Attempt artifacts saved.