A practical guide to using Gemini's spatial understanding capabilities for open-vocabulary object detection and image editing. The tutorial covers detecting visual objects in photos of books, magazines, and electronics using natural language prompts (no model training required), extracting bounding boxes and metadata via structured Pydantic outputs, and then using Gemini's image generation models (Nano Banana) to restore, straighten, and colorize the detected objects. Includes full Python code using the Google Gen AI SDK, with examples ranging from 15th-century woodcuts to modern circuit boards.
Table of contents
OverviewChallengeSetupDetecting visual objectsText extraction and dynamic labelingGeneralizing object detectionEditing visual objectsRestoring visual objectsColorizationCinematizationConclusionMore!3.7K Impressions