Benchmark report

Task verified

6m 51s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
One blocked host (browser-use.com CONNECT 403) before finding docs.browser-use.com/llms.txt and the V4 OpenAPI spec; nothing else cost a retry.

Where the time went

6:52 agent time
  • Research0:24
  • Attempt4:22
  • Verify2:05
6m 51sTotal elapsed
2:52Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/carry_cookie.py) and live responses support every criterion. POST /api/v4/profiles -> 201 id c7dbf64e-4600-480a-8e8c-9a7378655a7d (responses/01); POST /api/v4/browsers with {"profileId":...} -> 201 session A 87abde67-... (02) and later session B e53ab9b9-... (07), distinct ids, both bound to that profile in the saved request bodies. Playwright connected over each session's cdpUrl (https://<id>.cdp.browser-use.com, no query string) via chromium.connect_over_cdp(headers={'X-Browser-Use-API-Key': ...}); run_output.txt logs 'connected (API key sent as X-Browser-Use-API-Key handshake header)' for both. Session A loaded httpbin.org/cookies/set?tpc=benchmark (final URL /cookies, HTTP 200, body {"cookies":{"tpc":"benchmark"}}, responses/03); session A stopped via PATCH {"action":"stop"} -> 200 status stopped at 07:27:16.051 (04), confirmed by GET -> stopped (05), and GET /profiles -> cookieDomains ["httpbin.org"] (06) before session B was created at 07:27:16.34 (07). Session B's httpbin.org/cookies returned {"cookies":{"tpc":"benchmark"}} HTTP 200 (08), survived=True. Finally block: PATCH stop session B -> 200 stopped (09) and DELETE /profiles/{id} -> 204 (10); post-run checks show active browsers [] and GET profile -> 404. Docs fetched by the agent (cloud_guides_authentication.md: 'state is saved to the profile when it stops'; playwright-puppeteer-selenium.md: supply profileId and use the existing context) match the operations used. Printed console values match the saved JSON exactly, and an independent scan of code/, docs/, evidence/, responses*/ found only UUIDs, no credential (agent's own grep over the key also returned no match). No mocking: all responses come from api.browser-use.com and live httpbin page content read through the cloud browsers.

Attempt record

  1. 1:46Answered from saved key: Browser Use Cloud API keyno wait
  2. —First success verified

Attempt artifacts saved.