Benchmark report
Task verified
5m 11s elapsed
The saved attempt met the documented success criterion.
96/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- ADocs
- SCurrent
Clay's Routines docs name no concrete Clay-managed function routine_id (only the placeholder function:t_abc123), forcing a pivot to the Searches API.
Where the time went
5:12 agent time- Research0:23
- Attempt3:04
- Verify1:45
Judge reason
Saved code clay/enrich_stripe.sh makes two authenticated live calls to Clay's documented Searches API (POST /public/v0/search/query-mode and .../{search_id}/run, clay-api-key header). Saved headers show HTTP/2 200 with live CloudFront/request-id/etag and a period_quota block (used 571478, resets 2026-11-17), so responses are genuine, not mocked. 02_run_search_response.json matches the documented NextQueryModeCompanyResultsResponse shape and identifies the company: name Stripe, domain stripe.com, industry 'Technology, Information and Internet', size '5,001-10,000'. Stripe was identified by domain via the documented companies-query function clay.include_company_identifiers(("stripe.com")), per the live query reference saved in search_query_reference.md. run_output.txt prints exactly those values and matches the saved JSON. Grep of all saved artifacts and the transcript found no API key, access token or bearer token; the key was kept in /tmp/benchmark-private/clay.curlrc outside the home dir and used via curl -K. The agent also documented the Routines gap (no concrete Clay-managed function routine_id published; placeholder function:t_abc123) citing routines/clay-managed-functions.md and openapi.json rather than guessing an endpoint.
Attempt record
- 1:21Answered from saved key: Clay Public API keyno wait
- —First success verified
Attempt artifacts saved.