--- title: Deploying Large Language Models: vLLM and Quantization description: Step-by-step guide on how to accelerate large language models. Deployment of Large Language Models and the use of tools like vLLM and quantization. Measurement... canonical: https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Deploying Large Language Models: vLLM and Quantization | daily.dev og:description: Step-by-step guide on how to accelerate large language models. Deployment of Large Language Models and the use of tools like vLLM and quantization. Measurement... og:url: https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5 og:image: https://api.daily.dev/og/posts/jtkfcebF5.png og:image:alt: Deploying Large Language Models: vLLM and Quantization og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Deploying Large Language Models: vLLM and Quantization **[Towards Data Science](https://daily.dev/sources/tds)** · 9 min read · 1 upvotes · 0 comments ## Summary Step-by-step guide on how to accelerate large language models. Deployment of Large Language Models and the use of tools like vLLM and quantization. Measurement of latency and throughput. Installation and usage of vLLM. Comparison of vLLM and Hugging Face transformers. Deployment of A Large Language Model with vLLM. Benchmarking Latency and Throughput in Real Time. Quantization of Large Language Models and the use of BitsandBytes library. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Deploying Large Language Models: vLLM and Quantization","url":"https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5"},"datePublished":"2024-04-16T07:00:59.439Z","dateModified":"2024-05-09T08:39:06.345Z","description":"Step-by-step guide on how to accelerate large language models. Deployment of Large Language Models and the use of tools like vLLM and quantization. Measurement...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5211901fdc1643b55182a4f2dc5b3e65?_a=AQAEufR","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5211901fdc1643b55182a4f2dc5b3e65?_a=AQAEufR","isAccessibleForFree":true,"articleSection":"Towards Data Science","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Towards Data Science","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/tds","url":"https://daily.dev/sources/tds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/deploying-large-language-models-vllm-and-quantization-jtkfcebf5","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,llm","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Towards Data Science","item":"https://daily.dev/sources/tds"},{"@type":"ListItem","position":3,"name":"Deploying Large Language Models: vLLM and Quantization"}]} ```