--- title: xAI open-sourced X's "For You" algorithm — full pipeline... description: xAI has released the source code for X's (Twitter's) 'For You' recommendation algorithm under Apache-2.0 license. The pipeline breaks into retrieval... canonical: https://daily.dev/posts/xai-open-sourced-x-s-for-you-algorithm-full-pipeline-breakdown-zuvvcz2hh twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: xAI open-sourced X's "For You" algorithm — full pipeline breakdown | daily.dev og:description: xAI has released the source code for X's (Twitter's) 'For You' recommendation algorithm under Apache-2.0 license. The pipeline breaks into retrieval... og:url: https://daily.dev/posts/xai-open-sourced-x-s-for-you-algorithm-full-pipeline-breakdown-zuvvcz2hh og:image: https://api.daily.dev/og/posts/zuVVcz2Hh.png og:image:alt: xAI open-sourced X's "For You" algorithm — full pipeline breakdown og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # xAI open-sourced X's "For You" algorithm — full pipeline breakdown **[Md. Shafin Ahmed](https://daily.dev/sources/u8ir8dacsyjsphydqqtq9)** · [@mdshafinahmed](https://daily.dev/mdshafinahmed) · 1 min read · 2 upvotes · 0 comments ## Summary xAI has released the source code for X's (Twitter's) 'For You' recommendation algorithm under Apache-2.0 license. The pipeline breaks into retrieval (in-network via thunder, out-of-network vector search via phoenix, cluster-based via simclusters), scoring (multi-action prediction combining JAX and Rust, with a final weighted score and a determinantal point process reranker for diversity), a fully decoupled visibility/trust-and-safety layer (rule-based ALLOW/INTERSTITIAL/DROP filtering with fast-fail logic), and content understanding modules for text, image, and unsafe-content classification. The design separates ranking from moderation, prevents candidates from attending to each other during inference for consistent caching, and includes runnable training code rather than just inference stubs, making it a notable reference architecture for candidate generation and multi-objective ranking systems. ## Content ##  🔗[ https://github.com/xai-org/x-algorithm](https://github.com/xai-org/x-algorithm) 📜 Apache-2.0 ━━━━━━━━━━━━━━━━━━━━ ⚡ 𝗥𝗘𝗧𝗥𝗜𝗘𝗩𝗔𝗟 → thunder/ — in-network, in-memory → phoenix/ (retrieval) — vector-based out-of-network → simclusters/ — cluster-similarity candidates ⚡ 𝗦𝗖𝗢𝗥𝗜𝗡𝗚 → phoenix/ (ranking) — multi-action prediction, JAX + Rust serving → ranking_scorer.rs — Final Score = Σ(weight × P(action)) → vm-ranker/ — reranks via determinantal point process for diversity ⚡ 𝗩𝗜𝗦𝗜𝗕𝗜𝗟𝗜𝗧𝗬 (fully decoupled from ranking) → visibility-filtering/ — ALLOW / INTERSTITIAL / DROP → scarecrow/ + botmaker/ — rule engine for labeling → agatha/, bdsm/, user-cred-v2/ — account trust signals ⚡ 𝗖𝗢𝗡𝗧𝗘𝗡𝗧 𝗨𝗡𝗗𝗘𝗥𝗦𝗧𝗔𝗡𝗗𝗜𝗡𝗚 → grox/ — text/media classifiers → clip/ — image-text embeddings → media-model-proxy/ — adult/violence/gore detection ━━━━━━━━━━━━━━━━━━━━ 🔥 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀: • Ranking + moderation are architecturally separate — clean, auditable design • Candidates can't attend to each other during inference → consistent, cacheable scores • Ships real, runnable Phoenix training code (not just inference stubs) • First matching DROP rule ends visibility evaluation — fast-fail design 📌 Solid reference architecture for anyone building candidate generation, multi-objective ranking, or trust & safety systems. #opensource #machinelearning #recommendersystems #systemdesign #rust #jax #ai #backend ## Similar posts on daily.dev - [X open sources its algorithm while facing a transparency fine and Grok controversies](https://daily.dev/posts/x-open-sources-its-algorithm-while-facing-a-transparency-fine-and-grok-controversies-gx3cs4omq) · TechCrunch · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#open-source](https://daily.dev/tags/open-source), [#career](https://daily.dev/tags/career), [#rust](https://daily.dev/tags/rust) [View this post on daily.dev](https://daily.dev/posts/xai-open-sourced-x-s-for-you-algorithm-full-pipeline-breakdown-zuvvcz2hh) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/xai-open-sourced-x-s-for-you-algorithm-full-pipeline-breakdown-zuvvcz2hh","headline":"xAI open-sourced X's \"For You\" algorithm — full pipeline breakdown","text":"xAI has released the source code for X's (Twitter's) 'For You' recommendation algorithm under Apache-2.0 license. The pipeline breaks into retrieval (in-network via thunder, out-of-network vector search via phoenix, cluster-based via simclusters), scoring (multi-action prediction combining JAX and Rust, with a final weighted score and a determinantal point process reranker for diversity), a fully decoupled visibility/trust-and-safety layer (rule-based ALLOW/INTERSTITIAL/DROP filtering with fast-fail logic), and content understanding modules for text, image, and unsafe-content classification. The design separates ranking from moderation, prevents candidates from attending to each other during inference for consistent caching, and includes runnable training code rather than just inference stubs, making it a notable reference architecture for candidate generation and multi-objective ranking systems.","url":"https://daily.dev/posts/xai-open-sourced-x-s-for-you-algorithm-full-pipeline-breakdown-zuvvcz2hh","datePublished":"2026-08-14T03:43:18.024Z","dateModified":"2026-09-13T21:34:12.756Z","author":{"@type":"Person","name":"Md. Shafin Ahmed","url":"https://daily.dev/mdshafinahmed","image":"https://media.daily.dev/image/upload/s---a0uwu1i--/f_auto/v1783046504/avatars/avatar_u8iR8DacSYjsphyDqQTQ9?_a=BAMAMicg0","description":"Let's code your Future ! ! !","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":2270}},"image":"https://media.daily.dev/image/upload/s--m5gVQPI1--/f_auto/v1786678998/posts/zuVVcz2Hh?_a=BAMAMicg0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/sources/u8ir8dacsyjsphydqqtq9","name":"Md. Shafin Ahmed"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Md. Shafin Ahmed","item":"https://daily.dev/sources/u8ir8dacsyjsphydqqtq9"},{"@type":"ListItem","position":3,"name":"xAI open-sourced X's \"For You\" algorithm — full pipeline breakdown"}]} ```