Benchmark report
Task verified
6m 51s elapsed
The saved attempt met the documented success criterion.
96/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- ADocs
- SCurrent
One blocked host (browser-use.com CONNECT 403) before finding docs.browser-use.com/llms.txt and the V4 OpenAPI spec; nothing else cost a retry.
Where the time went
6:52 agent time- Research0:24
- Attempt4:22
- Verify2:05
Judge reason
Saved code (code/carry_cookie.py) and live responses support every criterion. POST /api/v4/profiles -> 201 id c7dbf64e-4600-480a-8e8c-9a7378655a7d (responses/01); POST /api/v4/browsers with {"profileId":...} -> 201 session A 87abde67-... (02) and later session B e53ab9b9-... (07), distinct ids, both bound to that profile in the saved request bodies. Playwright connected over each session's cdpUrl (https://<id>.cdp.browser-use.com, no query string) via chromium.connect_over_cdp(headers={'X-Browser-Use-API-Key': ...}); run_output.txt logs 'connected (API key sent as X-Browser-Use-API-Key handshake header)' for both. Session A loaded httpbin.org/cookies/set?tpc=benchmark (final URL /cookies, HTTP 200, body {"cookies":{"tpc":"benchmark"}}, responses/03); session A stopped via PATCH {"action":"stop"} -> 200 status stopped at 07:27:16.051 (04), confirmed by GET -> stopped (05), and GET /profiles -> cookieDomains ["httpbin.org"] (06) before session B was created at 07:27:16.34 (07). Session B's httpbin.org/cookies returned {"cookies":{"tpc":"benchmark"}} HTTP 200 (08), survived=True. Finally block: PATCH stop session B -> 200 stopped (09) and DELETE /profiles/{id} -> 204 (10); post-run checks show active browsers [] and GET profile -> 404. Docs fetched by the agent (cloud_guides_authentication.md: 'state is saved to the profile when it stops'; playwright-puppeteer-selenium.md: supply profileId and use the existing context) match the operations used. Printed console values match the saved JSON exactly, and an independent scan of code/, docs/, evidence/, responses*/ found only UUIDs, no credential (agent's own grep over the key also returned no match). No mocking: all responses come from api.browser-use.com and live httpbin page content read through the cloud browsers.
Attempt record
- 1:46Answered from saved key: Browser Use Cloud API keyno wait
- —First success verified
Attempt artifacts saved.