--- title: ByteDance is training a 10 trillion parameter AI model description: ByteDance is training an AI model with up to 10 trillion parameters, placing it in the same range as Anthropic's estimated Mythos system at ~8 trillion. This... canonical: https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: ByteDance is training a 10 trillion parameter AI model | daily.dev og:description: ByteDance is training an AI model with up to 10 trillion parameters, placing it in the same range as Anthropic's estimated Mythos system at ~8 trillion. This... og:url: https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku og:image: https://api.daily.dev/og/posts/r4idKdRKU.png og:image:alt: ByteDance is training a 10 trillion parameter AI model og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # ByteDance is training a 10 trillion parameter AI model **[Collections](https://daily.dev/sources/collections)** · 2 min read · 21 upvotes · 3 comments ## Summary ByteDance is training an AI model with up to 10 trillion parameters, placing it in the same range as Anthropic's estimated Mythos system at ~8 trillion. This dwarfs other Chinese competitors: Moonshot AI's Kimi K3 at 2.8 trillion, and Meituan's LongCat-2.0 and DeepSeek's V4-Pro at 1.6 trillion. The model is currently in pre-training, which typically takes 3–6 months before fine-tuning and release. Direct comparisons with US models remain difficult since Anthropic and OpenAI don't publicly disclose parameter counts, and raw parameter counts don't fully reflect model capability. ## Content According to the Financial Times, ByteDance is currently pre-training an AI model with up to 10 trillion parameters. If accurate, that would make it comparable in scale to Anthropic's Mythos system, estimated at around 8 trillion parameters, and would dwarf every other Chinese model by a wide margin. For context: Moonshot AI's Kimi K3 has 2.8 trillion parameters. Meituan's LongCat-2.0 and DeepSeek's V4-Pro, which previously led China, sit at 1.6 trillion. A 10T model would be roughly six times larger than Kimi K3. The catch is that raw parameter count is a bit misleading here. This is almost certainly a Mixture-of-Experts (MoE) architecture, where only a fraction of parameters are active during any given computation. DeepSeek-V3 activates about 5.5% of its total parameters; V4-Pro drops to 3.1%; Kimi K3 routes through 16 of 896 experts. At 10 trillion total, the active parameter count is probably somewhere between 200 and 500 billion. The 10T figure is essentially a memory bill, not a compute bill. The actual training compute works out to roughly 2×10²⁵ to 1.2×10²⁶ FLOPs, depending on token count (estimated at 15–40 trillion tokens). ByteDance has reportedly secured around 36,000 Blackwell GPUs through a Malaysian cloud operator, a deal reportedly structured to stay within US export controls. That's about 500 GB200 racks at an estimated cost of $2.5 billion. Pre-training is expected to take three to six months before fine-tuning and release. Direct comparisons with US frontier models remain difficult since Anthropic and OpenAI don't publicly disclose parameter counts. The FT also noted that ByteDance has avoided distilling rival models for over a year, preferring to develop independently. That matters: if this run succeeds, it would demonstrate that ByteDance can execute frontier-scale pre-training without using a competitor's model as a teacher. Some observers who had predicted a 10T Chinese model wouldn't arrive until early-to-mid 2027 are now openly reconsidering that timeline. ByteDance's track record in video generation suggests it's worth taking the report seriously. ## Questions this post answers ### How many parameters does ByteDance's new AI model reportedly have compared to other Chinese models like Kimi K3? ByteDance is reportedly pre-training a model with up to 10 trillion total parameters, according to the Financial Times, roughly six times larger than Moonshot AI's Kimi K3 at 2.8 trillion parameters, and far ahead of Meituan's LongCat-2.0 and DeepSeek's V4-Pro, both at 1.6 trillion. This would rival Anthropic's estimated 8 trillion-parameter Mythos system. _Anyone tracking the frontier model race can follow scaling comparisons like this on daily.dev._ ### Why doesn't a 10 trillion parameter Mixture-of-Experts model require 10 trillion parameters worth of compute? Because Mixture-of-Experts architectures only activate a fraction of total parameters per computation. DeepSeek-V3 activates about 5.5% of its parameters, V4-Pro drops to 3.1%, and Kimi K3 routes through 16 of 896 experts. For a 10 trillion parameter model, active parameters during inference are estimated at 200-500 billion, making the total count mostly a memory measure rather than a compute measure. _Developers weighing MoE trade-offs can dig deeper into architecture debates like this on daily.dev._ ### What hardware is ByteDance reportedly using to train its large-scale AI model? ByteDance has reportedly secured around 36,000 Nvidia Blackwell GPUs through a Malaysian cloud operator, structured to stay within US export controls, amounting to about 500 GB200 racks at an estimated cost of $2.5 billion. Estimated training compute ranges from 2×10²⁵ to 1.2×10²⁶ FLOPs, depending on a token count of 15-40 trillion, with pre-training expected to take three to six months. _Engineers following AI infrastructure and export-control workarounds can track developments like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion across x (as of 2026-09-13). **TL;DR:** There is no community discussion available to analyze — no comments were provided for this post. **Sentiment:** 50% positive · 30% mixed · 20% skeptical **By community** - x (mixed): No replies were available to assess sentiment. **Source threads** - [x](https://x.com/rohanpaul_ai/status/2086293817117036738) · 0 points · 0 comments ## Community discussion Top comments from developers on daily.dev. **@terrymunro** · 3 upvotes > > Pre-training typically takes three to six months before fine-tuning and release, so we’re probably looking at a late 2025 debut if things go smoothly. > > ByteDance also training a time machine. **@kashifmehmood68** · 0 upvotes > There is Longcat 2.0 already trained without nvidia on local chips only [https://pub.towardsai.net/chinas-longcat-2-0-is-a-1-6t-model-trained-without-nvidia-705ecac7c994?postPublishedType=repub](https://pub.towardsai.net/chinas-longcat-2-0-is-a-1-6t-model-trained-without-nvidia-705ecac7c994?postPublishedType=repub) --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#anthropic](https://daily.dev/tags/anthropic) [View this post on daily.dev](https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"ByteDance is training a 10 trillion parameter AI model","url":"https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku"},"datePublished":"2026-08-07T06:01:13.007Z","dateModified":"2026-09-13T19:44:22.382Z","description":"ByteDance is training an AI model with up to 10 trillion parameters, placing it in the same range as Anthropic's estimated Mythos system at ~8 trillion. This...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d6a614baf80be14a08a077437d6ae751?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d6a614baf80be14a08a077437d6ae751?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":3,"discussionUrl":"https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":21},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":3}],"keywords":"machine-learning,llm,anthropic","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"ByteDance is training a 10 trillion parameter AI model"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku","comment":[{"@type":"Comment","text":"Pre-training typically takes three to six months before fine-tuning and release, so we’re probably looking at a late 2025 debut if things go smoothly.\n\nByteDance also training a time machine.","datePublished":"2026-08-07T10:09:09.319Z","url":"https://daily.dev/posts/r4idKdRKU#c-io05LxSq2","author":{"@type":"Person","name":"Terence Munro","url":"https://daily.dev/terrymunro","image":"https://avatars2.githubusercontent.com/u/2586778?v=4"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3}},{"@type":"Comment","text":"There is Longcat 2.0 already trained without nvidia on local chips only https://pub.towardsai.net/chinas-longcat-2-0-is-a-1-6t-model-trained-without-nvidia-705ecac7c994?postPublishedType=repub","datePublished":"2026-08-10T10:37:42.201Z","url":"https://daily.dev/posts/r4idKdRKU#c-lrCRfS1rG","author":{"@type":"Person","name":"Kashif Mehmood","url":"https://daily.dev/kashifmehmood68","image":"https://lh3.googleusercontent.com/a/ACg8ocKuW0vI1DNJHVLKeajNn3CJUyBcjZoq1S02PWQnO7mc-UxHABA=s96-c"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku#faq","mainEntity":[{"@type":"Question","name":"How many parameters does ByteDance's new AI model reportedly have compared to other Chinese models like Kimi K3?","acceptedAnswer":{"@type":"Answer","text":"ByteDance is reportedly pre-training a model with up to 10 trillion total parameters, according to the Financial Times, roughly six times larger than Moonshot AI's Kimi K3 at 2.8 trillion parameters, and far ahead of Meituan's LongCat-2.0 and DeepSeek's V4-Pro, both at 1.6 trillion. This would rival Anthropic's estimated 8 trillion-parameter Mythos system. Anyone tracking the frontier model race can follow scaling comparisons like this on daily.dev."}},{"@type":"Question","name":"Why doesn't a 10 trillion parameter Mixture-of-Experts model require 10 trillion parameters worth of compute?","acceptedAnswer":{"@type":"Answer","text":"Because Mixture-of-Experts architectures only activate a fraction of total parameters per computation. DeepSeek-V3 activates about 5.5% of its parameters, V4-Pro drops to 3.1%, and Kimi K3 routes through 16 of 896 experts. For a 10 trillion parameter model, active parameters during inference are estimated at 200-500 billion, making the total count mostly a memory measure rather than a compute measure. Developers weighing MoE trade-offs can dig deeper into architecture debates like this on daily.dev."}},{"@type":"Question","name":"What hardware is ByteDance reportedly using to train its large-scale AI model?","acceptedAnswer":{"@type":"Answer","text":"ByteDance has reportedly secured around 36,000 Nvidia Blackwell GPUs through a Malaysian cloud operator, structured to stay within US export controls, amounting to about 500 GB200 racks at an estimated cost of $2.5 billion. Estimated training compute ranges from 2×10²⁵ to 1.2×10²⁶ FLOPs, depending on a token count of 15-40 trillion, with pre-training expected to take three to six months. Engineers following AI infrastructure and export-control workarounds can track developments like this on daily.dev."}}]} ```