--- title: Controlling Android with Gemini 3.7 Flash and 150 lines... description: A walkthrough demonstrates using Gemini 3.7 Flash's Computer Use capability to control an Android emulator via ADB with roughly 150 lines of Python. The agent... canonical: https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Controlling Android with Gemini 3.7 Flash and 150 lines of Python | daily.dev og:description: A walkthrough demonstrates using Gemini 3.7 Flash's Computer Use capability to control an Android emulator via ADB with roughly 150 lines of Python. The agent... og:url: https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk og:image: https://api.daily.dev/og/posts/PkoLRNckk.png og:image:alt: Controlling Android with Gemini 3.7 Flash and 150 lines of Python og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Controlling Android with Gemini 3.7 Flash and 150 lines of Python **[Philipp Schmid](https://daily.dev/sources/philschmid)** · 4 min read · 0 upvotes · 0 comments ## Summary A walkthrough demonstrates using Gemini 3.7 Flash's Computer Use capability to control an Android emulator via ADB with roughly 150 lines of Python. The agent takes screenshots, reasons about UI elements visually without accessibility trees, and emits normalized coordinates for taps and text input. As a demo, it opens Chrome, dismisses popups, and plays Wordle, solving it in two guesses by reading tile colors. The author argues this screenshot-based approach avoids the fragility of traditional mobile test tools like Appium and Espresso that break when UI elements change, and works even on webviews and canvas apps with no accessibility nodes. Full code is open sourced on GitHub alongside links to the Interactions API and Computer Use documentation. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I control an Android emulator using Gemini 3.7 Flash without accessibility IDs or XPath selectors? An agent loop captures screenshots via ADB (adb exec-out screencap -p), sends them to Gemini 3.7 Flash, which returns normalized 0-999 coordinates for taps, text input, or key presses. The Python script scales those coordinates to physical pixel dimensions and executes them with adb shell input tap, then loops with a fresh screenshot after each action. _daily.dev surfaces hands-on agent tutorials like this for developers building screenshot-driven automation._ ### Why do traditional mobile testing tools like Appium and Espresso break so often? They rely on accessibility trees, element IDs, and XPath selectors, so any UI change like an A/B test, redesigned checkout card, or new marketing banner breaks the test script. Webviews and dynamic canvas games are worse because they often expose zero accessibility nodes to ADB, making tree-based automation unreliable for those interfaces. _Developers weighing test automation approaches can track alternatives to Appium and Espresso on daily.dev._ ## Similar posts on daily.dev - [Control an Android Phone with Gemini 3.5 Flash Computer Use](https://daily.dev/posts/control-an-android-phone-with-gemini-3-5-flash-computer-use-rj19qu358) · Philipp Schmid · 0 upvotes · 0 comments - [Introducing computer use in Gemini 3.5 Flash](https://daily.dev/posts/introducing-computer-use-in-gemini-3-5-flash-m2isppphd) · DeepMind · 0 upvotes · 0 comments - [Gemini 3.5 Flash can now see and control your screen, and Google wants enterprises to trust it](https://daily.dev/posts/gemini-3-5-flash-can-now-see-and-control-your-screen-and-google-wants-enterprises-to-trust-it-ykketkrpx) · The Next Web · 1 upvotes · 0 comments - [AI, Meet App: Testing Android App Functions](https://daily.dev/posts/ai-meet-app-testing-android-app-functions-ryzubisjv) · ProAndroidDev · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#android](https://daily.dev/tags/android), [#google-gemini](https://daily.dev/tags/google-gemini) [View this post on daily.dev](https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Controlling Android with Gemini 3.7 Flash and 150 lines of Python","url":"https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk"},"datePublished":"2026-08-17T13:19:11.354Z","dateModified":"2026-08-17T23:28:51.215Z","description":"A walkthrough demonstrates using Gemini 3.7 Flash's Computer Use capability to control an Android emulator via ADB with roughly 150 lines of Python. The agent...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2a367f8c1ec2aafaeb655e7646b40277?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2a367f8c1ec2aafaeb655e7646b40277?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Philipp Schmid","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Philipp Schmid","logo":"https://media.daily.dev/image/upload/s---qdk89pI--/f_auto,q_auto/v1780213206/logos/philschmid?_a=BAMAMiWQ0","url":"https://daily.dev/sources/philschmid"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,android,google-gemini","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Philipp Schmid","item":"https://daily.dev/sources/philschmid"},{"@type":"ListItem","position":3,"name":"Controlling Android with Gemini 3.7 Flash and 150 lines of Python"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/controlling-android-with-gemini-3-7-flash-and-150-lines-of-python-pkolrnckk#faq","mainEntity":[{"@type":"Question","name":"How can I control an Android emulator using Gemini 3.7 Flash without accessibility IDs or XPath selectors?","acceptedAnswer":{"@type":"Answer","text":"An agent loop captures screenshots via ADB (adb exec-out screencap -p), sends them to Gemini 3.7 Flash, which returns normalized 0-999 coordinates for taps, text input, or key presses. The Python script scales those coordinates to physical pixel dimensions and executes them with adb shell input tap, then loops with a fresh screenshot after each action. daily.dev surfaces hands-on agent tutorials like this for developers building screenshot-driven automation."}},{"@type":"Question","name":"Why do traditional mobile testing tools like Appium and Espresso break so often?","acceptedAnswer":{"@type":"Answer","text":"They rely on accessibility trees, element IDs, and XPath selectors, so any UI change like an A/B test, redesigned checkout card, or new marketing banner breaks the test script. Webviews and dynamic canvas games are worse because they often expose zero accessibility nodes to ADB, making tree-based automation unreliable for those interfaces. Developers weighing test automation approaches can track alternatives to Appium and Espresso on daily.dev."}}]} ```