A tutorial demonstrates how to stream structured output from LLMs using the Embabel Agentic AI Framework in Java, contrasting it with Spring AI and LangChain4j, which only support raw text streaming and not native parsing into typed objects. It walks through four progressively complex scenarios using Anthropic's claude-sonnet-4-5 model: streaming a single object, multiple objects, objects with reasoning (thinking blocks), and objects combined with tool calls and reasoning. Embabel's createObjectStream() and createObjectStreamWithThinking() APIs handle buffering, JSON parsing, and separating reasoning from structured objects automatically, removing boilerplate developers would otherwise need to write themselves.
Table of contents
1. Overview2. Structured Streaming: Framework Limitations3. Setup4. Streaming a Single Object5. Streaming Multiple Objects6. Streaming with Reasoning7. Streaming with Tools and Reasoning8. ConclusionQuestions this post answers
Can Spring AI's ChatClient entity() method be used with the reactive stream() method to get typed objects?
No, Spring AI's entity() method for converting model output into a Java object only works with the blocking call() method, not with stream(). The ChatClient API reference states a convenience method for returning a Java entity with the reactive stream() method is planned for the future, and recommends using the Structured Output Converter in the meantime. Developers wiring up structured LLM streaming in Spring AI can track framework gaps like this via daily.dev.
How do I stream typed Java objects with reasoning and tool calls using Embabel?
Use createObjectStreamWithThinking() from StreamingPromptRunnerBuilder, which returns a Flux of StreamingEvent wrapping typed objects and reasoning fragments. Check event.isObject() and event.isThinking() in doOnNext() to separate parsed objects from reasoning text. When tools are registered via withToolObject(), the model calls tools first, reasons over results, then emits objects, followed by a reasoning summary. Teams building agentic Java apps with reasoning and tool calls can compare approaches like this on daily.dev.
Does LangChain4j's AiServices support returning custom POJOs when streaming LLM responses?
No, LangChain4j's AiServices only allows returning custom POJOs, lists, and enums for non-streaming calls. For streaming, the return type must be a TokenStream, which provides raw text as it arrives plus callback hooks, but does not provide typed objects as stream output. Java developers evaluating LangChain4j versus alternatives for structured streaming can dig deeper on daily.dev.