--- title: "ByteDance is training a 10 trillion parameter AI model" url: https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku source_url: https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku type: collection source: "Collections" published: 2026-08-07T06:01:13.007Z updated: 2026-08-09T03:30:46.911Z tags: ["machine-learning", "llm", "anthropic"] reading_time: 2 upvotes: 21 comments: 3 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # ByteDance is training a 10 trillion parameter AI model **[Collections](https://daily.dev/sources/collections)** · 2 min read · 21 upvotes · 3 comments ## Summary ByteDance is training an AI model with up to 10 trillion parameters, placing it in the same range as Anthropic's estimated Mythos system at ~8 trillion. This dwarfs other Chinese competitors: Moonshot AI's Kimi K3 at 2.8 trillion, and Meituan's LongCat-2.0 and DeepSeek's V4-Pro at 1.6 trillion. The model is currently in pre-training, which typically takes 3–6 months before fine-tuning and release. Direct comparisons with US models remain difficult since Anthropic and OpenAI don't publicly disclose parameter counts, and raw parameter counts don't fully reflect model capability. ## Content According to the Financial Times, ByteDance is currently pre-training an AI model with up to 10 trillion parameters. If accurate, that would make it comparable in scale to Anthropic's Mythos system, estimated at around 8 trillion parameters, and would dwarf every other Chinese model by a wide margin. For context: Moonshot AI's Kimi K3 has 2.8 trillion parameters. Meituan's LongCat-2.0 and DeepSeek's V4-Pro, which previously led China, sit at 1.6 trillion. A 10T model would be roughly six times larger than Kimi K3. The catch is that raw parameter count is a bit misleading here. This is almost certainly a Mixture-of-Experts (MoE) architecture, where only a fraction of parameters are active during any given computation. DeepSeek-V3 activates about 5.5% of its total parameters; V4-Pro drops to 3.1%; Kimi K3 routes through 16 of 896 experts. At 10 trillion total, the active parameter count is probably somewhere between 200 and 500 billion. The 10T figure is essentially a memory bill, not a compute bill. The actual training compute works out to roughly 2×10²⁵ to 1.2×10²⁶ FLOPs, depending on token count (estimated at 15–40 trillion tokens). ByteDance has reportedly secured around 36,000 Blackwell GPUs through a Malaysian cloud operator, a deal reportedly structured to stay within US export controls. That's about 500 GB200 racks at an estimated cost of $2.5 billion. Pre-training is expected to take three to six months before fine-tuning and release. Direct comparisons with US frontier models remain difficult since Anthropic and OpenAI don't publicly disclose parameter counts. The FT also noted that ByteDance has avoided distilling rival models for over a year, preferring to develop independently. That matters: if this run succeeds, it would demonstrate that ByteDance can execute frontier-scale pre-training without using a competitor's model as a teacher. Some observers who had predicted a 10T Chinese model wouldn't arrive until early-to-mid 2027 are now openly reconsidering that timeline. ByteDance's track record in video generation suggests it's worth taking the report seriously. ## Questions this post answers ### How many parameters does ByteDance's new AI model have compared to other Chinese AI models? ByteDance's model targets up to 10 trillion parameters, far ahead of other Chinese competitors. Moonshot AI's Kimi K3 sits at 2.8 trillion, while Meituan's LongCat-2.0 and DeepSeek's V4-Pro previously led the Chinese field at 1.6 trillion. The model is currently in pre-training, which typically takes three to six months before fine-tuning and release. _Developers tracking the LLM landscape and competitive parameter counts follow developments like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion across x (as of 2026-08-09). **TL;DR:** There is no community discussion available to analyze — no comments were provided for this post. **Sentiment:** 50% positive · 30% mixed · 20% skeptical **By community** - x (mixed): No replies were available to assess sentiment. **Source threads** - [x](https://x.com/rohanpaul_ai/status/2086293817117036738) · 0 points · 0 comments ## Community discussion Top comments from developers on daily.dev. **@terrymunro** · 3 upvotes > > Pre-training typically takes three to six months before fine-tuning and release, so we’re probably looking at a late 2025 debut if things go smoothly. > > ByteDance also training a time machine. **@kashifmehmood68** · 0 upvotes > There is Longcat 2.0 already trained without nvidia on local chips only [https://pub.towardsai.net/chinas-longcat-2-0-is-a-1-6t-model-trained-without-nvidia-705ecac7c994?postPublishedType=repub](https://pub.towardsai.net/chinas-longcat-2-0-is-a-1-6t-model-trained-without-nvidia-705ecac7c994?postPublishedType=repub) --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#anthropic](https://daily.dev/tags/anthropic) [View this post on daily.dev](https://daily.dev/posts/bytedance-is-training-a-10-trillion-parameter-ai-model-r4idkdrku)