--- title: Petals – Run LLMs at home, BitTorrent-style | daily.dev description: Petals is an open-source project that lets you run large language models (Llama 3.1 up to 405B, Mixtral 8x22B, Falcon 180B, BLOOM 176B) on consumer-grade GPUs... canonical: https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Petals – Run LLMs at home, BitTorrent-style | daily.dev og:description: Petals is an open-source project that lets you run large language models (Llama 3.1 up to 405B, Mixtral 8x22B, Falcon 180B, BLOOM 176B) on consumer-grade GPUs... og:url: https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u og:image: https://api.daily.dev/og/posts/z1p2lcu2u.png og:image:alt: Petals – Run LLMs at home, BitTorrent-style og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Petals – Run LLMs at home, BitTorrent-style **[Hacker News](https://daily.dev/sources/hn)** · 1 min read · 0 upvotes · 0 comments ## Summary Petals is an open-source project that lets you run large language models (Llama 3.1 up to 405B, Mixtral 8x22B, Falcon 180B, BLOOM 176B) on consumer-grade GPUs or Google Colab using a BitTorrent-style distributed network. Each participant loads a portion of the model and joins a peer network serving the remaining layers. Single-batch inference reaches up to 6 tokens/sec for Llama 2 70B, sufficient for chatbots and interactive apps. Unlike standard LLM APIs, Petals exposes full PyTorch and Hugging Face Transformers flexibility, enabling custom fine-tuning, sampling methods, and access to hidden states. It is part of the BigScience research workshop. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 4 discussions and 312 comments across hackernews, lobsters (as of 2026-07-23). **TL;DR:** HN commenters find Petals genuinely interesting as a way to run very large models on consumer hardware, but raise serious concerns about network latency penalties vs. local inference, the impracticality of distributed fine-tuning, security against malicious nodes, and prompt privacy — while a large portion of the thread drifts into crypto/Filecoin debates. **Sentiment:** 35% positive · 45% mixed · 20% skeptical **The case for** - The distributed layer-splitting approach is a practical solution for users who lack enough VRAM to run 70B+ models locally at all. - Pipelining requests across nodes can achieve reasonable GPU utilization even with heterogeneous hardware. - A non-crypto, kudos-style incentive system (similar to AI Horde) is seen as a pragmatic and community-friendly choice. - The project exposes full PyTorch flexibility, enabling custom sampling and fine-tuning workflows not available via standard APIs. **The pushback** - Network latency between geographically dispersed nodes significantly hurts throughput compared to local inference (llama.cpp on 2×RTX 3090 can hit ~20 t/s vs. Petals' ~6 t/s). - Distributed fine-tuning over unreliable consumer networks is energy/cost-inefficient; cloud alternatives (Lambda, Replicate) are cheaper for the same task. - No clear defense against malicious nodes returning junk results or subtly corrupting gradients during training. - Prompt privacy is unaddressed — nodes can log all prompts passing through them (Sybil farm risk). - Electricity costs for running a GPU at full load (~$450/year for a 4090) make altruistic contribution economically unattractive without real compensation. - The centralized incentive tracker is a single point of failure that contradicts the otherwise decentralized design. **By community** - hackernews (mixed): Commenters appreciate the concept for hardware-constrained users but are skeptical about latency, fine-tuning practicality, security, and privacy, with much of the thread diverted into crypto/Filecoin tangents. - lobsters (mixed): No comments were present, so no community signal is available beyond the 5-point submission. **Hottest debate:** Whether monetary/crypto incentives would help or harm a volunteer GPU-sharing network, with a long sub-thread using Filecoin as a case study for why for-profit distributed resource sharing tends to fail. **Open questions** - How does Petals defend against malicious nodes that return corrupted layer outputs or subtly sabotage gradient updates during fine-tuning? - Can the system enforce any form of prompt privacy or zero-data-retention on participating nodes? - At what network bandwidth threshold does distributed inference become competitive with local quantized inference? - Will the centralized incentive tracker become a bottleneck or censorship point as the network scales? **Highlights** > Hi, a Petals dev here. You're right, there's no point in using Petals if your machine has enough GPU memory to fit the model and you're okay with the quantization quality. We developed Petals for people who have less GPU memory than needed. Also, there's still a chance of larger open models being released in the future. > — [borzunov on hackernews](https://news.ycombinator.com/item?id=37551681) > Can't wait to run a Sybil farm of subsidized proxy nodes and log all the juicy prompts. > — [inigyou on hackernews · 2 comments](https://news.ycombinator.com/item?id=49016102) > How does this defend against a malicious participant altering the output of their share of the larger computation? Even without some kind of method for e.g. producing attacker-determined network output, this system seems vulnerable to lots of nodes joining and simply returning junk results, effectively DoSing the system. > — [malwrar on hackernews · 1 comments](https://news.ycombinator.com/item?id=37551145) > Finetuning in a distributed way with questionable network would be lot more energy/cost inefficient than doing it with a single node or a well connected cluster. Also, you can finetune 70b model for million tokens for $2 in lambda cloud or <$10 in replicate. > — [YetAnotherNick on hackernews](https://news.ycombinator.com/item?id=37554330) > I've occasionally written comments about the difficult of achieving proof-of-useful-work mechanisms and I should probably find or write a standard post about this question. In order to be as decentralized as Bitcoin and related PoW mechanisms, a proof-of-work mechanism should have * the ability to create an unbounded number of instances of the problem deterministically from numeric seeds * the ability to scale difficulty up and down using a difficulty parameter * in a way where anyone can easily confirm that a given problem instance corresponds to a given seed, and anyone can easily confirm that a given solution correctly solves a given problem instance I'm not aware of a proof-of-useful-work mechanism that meets these criteria. (I'm also not aware of an argument that it's impossible to have one. Zero-knowledge proofs might actually go far toward making it possible in the future. but it's kind of complicated.) > — [schoen on hackernews](https://news.ycombinator.com/item?id=49016617) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49015735) · 101 points · 62 comments - [hackernews](https://news.ycombinator.com/item?id=37546810) · 485 points · 250 comments - [hackernews](https://news.ycombinator.com/item?id=44106850) · 1 points · 0 comments - [lobsters](https://lobste.rs/s/qckbua/petals_run_llms_at_home_bittorrent_style) · 5 points · 0 comments ## Similar posts on daily.dev - [GitHub - lyogavin/airllm: AirLLM 70B inference with single 4GB GPU](https://daily.dev/posts/github---lyogavin-airllm-airllm-70b-inference-with-single-4gb-gpu-s8ukkihfl) · Hacker News · 3 upvotes · 1 comments - [I Ran Local LLMs on My Android Phone](https://daily.dev/posts/i-ran-local-llms-on-my-android-phone-3zszeolxe) · It's Foss · 8 upvotes · 0 comments - [Guide to Local LLMs in 2026: Privacy, Tools & Hardware](https://daily.dev/posts/guide-to-local-llms-in-2026-privacy-tools-hardware-yigh17bqi) · SitePoint · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#pytorch](https://daily.dev/tags/pytorch), [#transformers](https://daily.dev/tags/transformers) [View this post on daily.dev](https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Petals – Run LLMs at home, BitTorrent-style","url":"https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u"},"datePublished":"2026-07-23T02:57:53.985Z","dateModified":"2026-07-23T07:53:23.826Z","description":"Petals is an open-source project that lets you run large language models (Llama 3.1 up to 405B, Mixtral 8x22B, Falcon 180B, BLOOM 176B) on consumer-grade GPUs...","image":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","thumbnailUrl":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/petals-run-llms-at-home-bittorrent-style-z1p2lcu2u","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning,distributed-systems,pytorch,transformers","timeRequired":"PT1M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Petals – Run LLMs at home, BitTorrent-style"}]} ```