Amazon Bedrock AgentCore Browser now supports OS Level Actions, extending AI agent automation beyond the browser's web layer (DOM/CDP) to the full OS desktop. This enables agents to interact with native dialogs, security prompts, certificate choosers, right-click context menus, and keyboard shortcuts that Playwright and CDP cannot reach. The feature works via the InvokeBrowser API using an action-screenshot-reaction loop: the agent dispatches an action (mouse click, drag, scroll, key press, shortcut), captures a full-desktop screenshot, sends it to a vision model, and decides the next step. Eight actions are supported across three categories: mouse control (mouseClick, mouseMove, mouseDrag, mouseScroll), keyboard input (keyType, keyPress, keyShortcut), and visual capture (screenshot). No additional setup is required for existing browser configurations.

9m read timeFrom aws.amazon.com
Post cover image
Table of contents
How OS Level Actions workSupported actionsGetting startedPutting it together: dismissing a print dialogStop the session and clean upConclusion