---
title: "Deploy Falcon 180B on Amazon SageMaker"
url: https://daily.dev/posts/deploy-falcon-180b-on-amazon-sagemaker-1i4jahuae
source_url: https://www.philschmid.de/sagemaker-falcon-180b
type: article
source: "Philipp Schmid"
published: 2026-05-31T07:41:34.853Z
updated: 2026-05-31T09:33:14.948Z
tags: ["aws", "llm"]
reading_time: 7
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy Falcon 180B on Amazon SageMaker

**[Philipp Schmid](https://daily.dev/sources/philschmid)** · 7 min read · 0 upvotes · 0 comments

## Summary

Step-by-step guide to deploying Falcon 180B, a 180-billion-parameter open source LLM, on Amazon SageMaker using the Hugging Face LLM Deep Learning Container (DLC) powered by Text Generation Inference (TGI). Covers environment setup, retrieving the DLC image URI, hardware requirements (p4de.24xlarge with 8 A100 GPUs), deploying the model endpoint, running chat inference with configurable parameters, and building a streaming Gradio chatbot demo using SageMaker's Server-Sent-Events streaming support.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://www.philschmid.de/sagemaker-falcon-180b>

---

Tags: [#aws](https://daily.dev/tags/aws), [#llm](https://daily.dev/tags/llm)

[View this post on daily.dev](https://daily.dev/posts/deploy-falcon-180b-on-amazon-sagemaker-1i4jahuae)