A step-by-step guide to running Ollama LLMs locally on Fedora Linux using Podman containers. Covers creating a persistent volume for model storage, launching the Ollama container, pulling and running Llama 3, interacting with the built-in REST API, and basic container management commands. Also touches on optional Nvidia GPU passthrough for hardware acceleration.
Table of contents
What is Ollama?Verify or Install PodmanStep 1: Create a Persistent Volume for Your ModelsStep 2: Run the Ollama ContainerStep 3: Download and Run an AI ModelStep 4: Interact with the Local APIConclusion8 Impressions1 Comment