--- title: Building Multimodal Workflows with a Local LLM | daily.dev description: A step-by-step build of a multimodal LLM workflow using Gemma 4 (E4B variant) running locally via Ollama, combined with Pydantic for structured output. The... canonical: https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building Multimodal Workflows with a Local LLM | daily.dev og:description: A step-by-step build of a multimodal LLM workflow using Gemma 4 (E4B variant) running locally via Ollama, combined with Pydantic for structured output. The... og:url: https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po og:image: https://api.daily.dev/og/posts/fhpCpS3po.png og:image:alt: Building Multimodal Workflows with a Local LLM og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building Multimodal Workflows with a Local LLM **[Towards Data Science](https://daily.dev/sources/tds)** · 9 min read · 0 upvotes · 0 comments ## Summary A step-by-step build of a multimodal LLM workflow using Gemma 4 (E4B variant) running locally via Ollama, combined with Pydantic for structured output. The workflow analyzes travel photos, extracting structured records (scene summary, mood, objects, uncertainty notes) from images plus metadata, then synthesizes those records into a trip-level narrative summary. It also covers a practical compatibility fix: on Windows with Ollama 0.32.5, gemma4:e4b failed to process image inputs correctly, requiring a workaround using Unsloth's split GGUF files (quantized model plus multimodal projector) imported via a custom Modelfile. The workflow is extended into a small travel-memory app with photo timelines, maps, and search, with full code shared in a GitHub repo. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why doesn't gemma4:e4b process image inputs correctly in Ollama on Windows? On Ollama version 0.32.5 with gemma4:e4b on Windows, the model accepts multimodal requests but fails to actually use the visual content of images. A workaround is to load Gemma 4 from two separate Unsloth GGUF files, the quantized model and the mmproj-BF16.gguf multimodal projector, combined via a custom Modelfile and imported into Ollama under a new model name. _Developers hitting model-specific multimodal bugs can track Ollama and Gemma fixes on daily.dev._ ### How do I get structured JSON output from Ollama using Pydantic schemas? Pass the Pydantic model's model_json_schema() to the format parameter in ollama.chat(), then parse the returned response.message.content with model_validate_json() to get a typed Python object. This lets downstream code access LLM outputs through typed attributes instead of raw text, and works for both image-plus-text and text-only prompts. _daily.dev helps developers building structured LLM pipelines stay current on tooling like Ollama and Pydantic._ ## Similar posts on daily.dev - [From Local LLM to Tool-Using Agent](https://daily.dev/posts/from-local-llm-to-tool-using-agent-kmmln7rio) · Towards Data Science · 1 upvotes · 0 comments --- Tags: [#ollama](https://daily.dev/tags/ollama), [#gemma](https://daily.dev/tags/gemma), [#pydantic](https://daily.dev/tags/pydantic) [View this post on daily.dev](https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building Multimodal Workflows with a Local LLM","url":"https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po"},"datePublished":"2026-08-12T13:41:11.578Z","dateModified":"2026-08-12T13:41:39.155Z","description":"A step-by-step build of a multimodal LLM workflow using Gemma 4 (E4B variant) running locally via Ollama, combined with Pydantic for structured output. The...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1963ef59841b578a8a08980c95fd4f02?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1963ef59841b578a8a08980c95fd4f02?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Towards Data Science","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Towards Data Science","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/tds","url":"https://daily.dev/sources/tds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ollama,gemma,pydantic","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Towards Data Science","item":"https://daily.dev/sources/tds"},{"@type":"ListItem","position":3,"name":"Building Multimodal Workflows with a Local LLM"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-multimodal-workflows-with-a-local-llm-fhpcps3po#faq","mainEntity":[{"@type":"Question","name":"Why doesn't gemma4:e4b process image inputs correctly in Ollama on Windows?","acceptedAnswer":{"@type":"Answer","text":"On Ollama version 0.32.5 with gemma4:e4b on Windows, the model accepts multimodal requests but fails to actually use the visual content of images. A workaround is to load Gemma 4 from two separate Unsloth GGUF files, the quantized model and the mmproj-BF16.gguf multimodal projector, combined via a custom Modelfile and imported into Ollama under a new model name. Developers hitting model-specific multimodal bugs can track Ollama and Gemma fixes on daily.dev."}},{"@type":"Question","name":"How do I get structured JSON output from Ollama using Pydantic schemas?","acceptedAnswer":{"@type":"Answer","text":"Pass the Pydantic model's model_json_schema() to the format parameter in ollama.chat(), then parse the returned response.message.content with model_validate_json() to get a typed Python object. This lets downstream code access LLM outputs through typed attributes instead of raw text, and works for both image-plus-text and text-only prompts. daily.dev helps developers building structured LLM pipelines stay current on tooling like Ollama and Pydantic."}}]} ```