A step-by-step guide to running LLMs locally using Ollama and integrating them into a Spring Boot application via Spring AI 2.0. Covers project setup with Maven, configuration via application.yml, and using the ChatClient abstraction to send prompts to locally hosted models like Qwen or Llama. Highlights that the same ChatClient code works for both local Ollama and cloud providers (OpenAI, Anthropic), making it easy to switch without changing business logic. Includes practical next steps for experimenting with different models and temperature settings.
Table of contents
IntroductionWhat is Ollama?Spring AI and OllamaProject SetupConfigurationCalling the ModelRunning the ExampleNext StepsCloud or Local — Your ChoiceFurther Reading359 Impressions