Benchmark report

Task verified

5m 37s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Parsing Turso's JavaScript-rendered doc HTML with regex to locate the authorization query parameter took several extra passes; nothing else failed.

Where the time went

5:37 agent time
  • Research0:26
  • Attempt3:41
  • Verify1:31
5m 37sTotal elapsed
2:44Time to first success
0:00Waited on the owner

Judge reason

Saved run.py (stdlib http.client, no mocks) and work/out/transcript.json + run-console.log show a live end-to-end run against api.turso.tech: POST /v1/organizations/rickchen/databases created tpcbench-8232f45f (200, hostname ...aws-ap-northeast-1.turso.io); admin full-access token created table items and 2 rows; the scoped credential came from the documented mechanism POST /v1/organizations/{org}/databases/{db}/auth/tokens?authorization=read-only (doc quoted from docs.turso.tech/api-reference/databases/create-token, page fetched HTTP 200). Connecting with only that token over https /v2/pipeline (TLSv1.3, TLS_AES_128_GCM_SHA256, check_hostname true, verify_mode 2, peer CN = db hostname) returned 200 with both rows; the credential is identified by its SHA-256 fingerprint after SELECT current_user correctly returned SQLite 'no such column' (libSQL has no current_user); INSERT was refused with BLOCKED 'SQL write operations are forbidden (current session doesn't have write permission)' and an admin re-read confirmed count stayed 2. Revocation used the documented POST .../auth/rotate (invalidate-tokens) -> 200, after which the same token got HTTP 401 'role was invalidated after token was issued'. DELETE /v1/organizations/rickchen/databases/tpcbench-8232f45f -> 200 {"database":"tpcbench-8232f45f"}, follow-up GET -> 404 'record not found', and the name is absent from the databases list. Turso scopes auth tokens per database, and this database contains only table items, so the read-only token is a credential limited to reading items; the agent stated that path and its limitation. Secret hygiene verified: my own grep over all saved files found no JWT and no password-bearing string (only the API's empty "password": "" field); JWTs appear solely as fingerprints.

Attempt record

  1. 0:37Answered from saved key: TURSO_API_TOKENno wait
  2. —First success verified

Attempt artifacts saved.