A hands-on walkthrough builds a complete AI chat assistant inside a Java Spring Boot app using Spring AI 2.0, Vaadin for the UI, and no JavaScript. It covers streaming token-by-token responses, wiring up RAG with a vector store, tracking token costs via Micrometer, and handling failures gracefully. It also compares Spring AI against LangChain4j, explaining when to reach for each, and calls out that Spring AI 2.0 removed the QuestionAnswerAdvisor in favor of RetrievalAugmentationAdvisor. Several production gotchas are highlighted: Claude models reject sampling parameters like temperature, Anthropic lacks an embedding API, and setting an unmeasured similarityThreshold can silently break RAG retrieval with no errors.
Table of contents
What is Spring AI?Spring AI vs. LangChain4jWhat we're buildingStep 1: DependenciesStep 2: A ChatClient with a system promptStep 3: The chat view — a real UI in ~40 lines of JavaStep 4: Streaming, done properlyStep 5: RAG — make it answer from your docsProduction concernsFrequently asked questionsWhere to go from hereQuestions this post answers
Why does my Spring AI RAG application say it doesn't know the answer even though the documents are ingested?
The most common cause is a similarityThreshold on the document retriever set too high for the embedding model in use, which silently filters out every retrieved chunk. Retrieval then returns nothing, the model has no context, and it answers 'I don't know' with no error or warning. The fix is to start with topK alone, confirm documents come back, then add a threshold only as a measured value specific to that embedding model. Developers debugging silent RAG failures can track Spring AI gotchas like this on daily.dev.
What replaced QuestionAnswerAdvisor in Spring AI 2.0?
Spring AI 2.0 removed QuestionAnswerAdvisor, the advisor shown in most older RAG tutorials. It was replaced by RetrievalAugmentationAdvisor, found in the spring-ai-rag module, composed with a DocumentRetriever such as VectorStoreDocumentRetriever. The new approach is more verbose but makes retrieval parameters explicit and supports query transformers and post-processors via the same builder. Teams upgrading Spring AI RAG code can follow breaking API changes like this via daily.dev.
Why do my Spring AI calls to Claude fail with a temperature deprecated error?
Current Claude models reject sampling parameters outright, including temperature, top-p, and top-k, returning a 400 invalid_request_error stating temperature is deprecated for that model. The application compiles and starts fine, but every request fails at runtime once a chat call is made. The fix is to steer model behavior through the system prompt instead of sampling parameters. Java developers wiring Claude into Spring AI can catch runtime gotchas like this on daily.dev.