Browser & Computer-Use Agents
Table of Contents
Link copied!
These agents perceive a screen or DOM and act (click, type, scroll, navigate) to complete tasks a human would do in a browser or OS. Useful for automation over apps that lack APIs; still the least reliable and highest-risk agent category. Treat them as an intern who can be prompt-injected by any web page: sandbox, scope, and supervise.
Models / agents
| Agent | Best at | Watch out | Key fact |
|---|---|---|---|
| Claude for Chrome / computer use | Operating your browser (Chrome extension) or a virtual desktop via the API's computer-use tool, with permission prompts on sensitive actions | Anthropic explicitly warns about prompt-injection; access gated/rolled out to subscribers | Available to Max subscribers (Chrome) / via API (computer use) 🔄 |
| OpenAI computer-using agents | Multi-step web tasks in a hosted virtual browser, via ChatGPT agent and the computer-use tool / Responses API | Standalone "Operator" was consolidated into ChatGPT agent, so check current naming; supervise checkouts/logins | Pro/Plus feature + API 🔄 |
| Google Project Mariner / Gemini | Agentic browser tasks surfacing in Gemini and Chrome | Research-turned-product; capability still maturing | 🔄 |
Also: Perplexity Comet (agentic browser), The Browser Company's Dia, and other AI browsers.
Browser-automation infrastructure (for developers)
| Tool | Best at | Key fact |
|---|---|---|
| Browserbase | Running many concurrent, stealthy browser sessions with auth/proxy/captcha handling; the infra layer under browser agents | Managed headless-browser cloud; usage-based, free tier to start 🔄 |
| Stagehand | Mixing deterministic Playwright with natural-language act/extract/observe steps, for reliability of code plus flexibility of an agent |
Browserbase's open-source (MIT) SDK on Playwright; pairs with Browserbase |
Also: Playwright (the deterministic automation baseline, now with AI/MCP add-ons), Browser Use (popular open-source browser-agent library), Skyvern (vision-based automation), Notte, Steel.dev (agent browser infra).
Reliability & safety caveats (do not skip):
- Prompt injection is the core threat. A malicious page can hijack an agent that reads it. Never give a browser agent standing access to email, banking, or admin without human confirmation on each sensitive action.
- Reliability is task-dependent. Success rates on clean, well-known flows are high. Long-tail UIs, popups, and captchas still break agents. Build retries and human fallbacks.
- Prefer APIs when they exist. Computer-use is a fallback for apps with no API, not a first choice. Prefer deterministic (Playwright/Stagehand) steps for known flows and let the model handle only the fuzzy parts.
- Isolate. Run in a sandboxed/ephemeral browser (Browserbase-style), scope credentials, and log everything.