Benchmark report

Task verified

4m 5s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The undocumented org_id requirement on POST /projects caused a 400 and two extra steps (list orgs, patch the script) before the run succeeded.

Where the time went

4:05 agent time
  • Starting0:01
  • Research0:20
  • Attempt2:14
  • Verify1:30
4m 5sTotal elapsed
1:29Time to first success
0:00Waited on the owner

Judge reason

Saved script work/run_neon_task.sh ran live against https://console.neon.tech/api/v2: POST /projects -> HTTP 201 creating project super-smoke-54902114 (name tpcbench-bf7f2775a46d7d7e, region aws-us-east-2), readiness polled until unfinished_operations went 3 -> 0. psql (PostgreSQL) 16.15 connected with sslmode=verify-full&sslrootcert=system (03_connection_params.redacted.txt, 04_psql_command.redacted.txt) and 05_psql_select1.txt shows select_1 = 1, server_version 17.11, psql exit code 0; pg_stat_ssl ssl=f is the expected Neon-proxy artifact the criteria allow. Cleanup verified: DELETE /projects/super-smoke-54902114 -> HTTP 200 with the full project body, follow-up GET -> HTTP 404 {"message":"project not found"} (request_id 3ee3522e-...), and GET /projects shows still_listed=0 with an empty project_ids list. No secrets leaked: connection_uri is [REDACTED_CONNECTION_URI] and every password field is [REDACTED] in 01_create_project.redacted.json; my own grep over work/outputs, evidence.txt and result.json found no password or key material, and the agent's grep for the key value across saved files returned no match. Responses carry live request_ids, timestamps and account metadata consistent with real API calls; nothing is mocked or hard-coded.

Attempt record

  1. 0:34Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.