Meta's engineering team presents two architectural breakthroughs for ads ranking at scale: a multi-stage sequence model that decouples heavy offline user modeling from lightweight online ranking, and a dense tokenization with target-aware attention technique. The offline stage processes thousands of user events asynchronously using deep transformers, caching embeddings that the online stage combines with real-time signals. This separation allows model complexity to scale without proportional serving cost increases. Notably, the system exhibits LLM-style log-linear scaling laws despite working with sparse ID features rather than dense text. Four scaling levers are identified: balanced model shape, multi-stage tunability, sequence diversity, and semantic feature enrichment. The architecture, a core part of Meta's GEM platform, has driven cumulative lifts of 6% in Instagram conversions, 3% in Facebook conversions, and 3.5% in Facebook ad clicks.
