Vision LLMs like GPT-4.1 can parse PDF pages as images, extracting not just text and tables but also the content of charts and diagrams — something traditional text-based parsers (PyMuPDF, Azure Layout, Docling) cannot do. The article demonstrates a `parse_page_vision` function using Pydantic structured output to return page markdown plus per-figure descriptions and transcriptions. Key trade-offs: vision parsing is slower, more expensive per page, and less numerically exact than deterministic parsers, and model quality matters significantly (GPT-4.1 finds all charts where GPT-4o-mini misses half). The recommended strategy is to use vision parsing selectively for visually dense pages where text parsers return empty results. Mistral Document AI is also mentioned as a packaged alternative, with a noted limitation around missing bounding boxes for text content.

14m read timeFrom towardsdatascience.com
Post cover image
Table of contents
1. The one thing only a vision model can do: make an image searchable2. It also parses text and tables, like the others3. The model matters: gpt-4o-mini misses charts that gpt-4.1 reads4. The honest trade: exactness and cost5. How it works: parse_page_vision6. The lighter mode: ask the page directly7. Four parsers now, one of them reads the pictures8. Conclusion9. Sources and further reading
152 Impressions