SingleStore
Read post

Identity resolution without the nightly batch

Nightly batch rebuilds of identity graphs in advertising platforms introduce hours of latency that silently corrupt attribution, frequency capping, and consent propagation. When a household's devices are only linked during a 2 AM rebuild, same-day conversions get misattributed, users receive duplicate ads across devices, and GDPR opt-outs remain active for up to 17 hours — all documented in the platform's own audit logs. The standard match-rate metric is circular and masks this staleness. As probabilistic edges grow and graphs scale to billions of nodes, batch windows stretch past business hours, compounding the problem. The proposed alternative is continuous, incremental resolution using data sharding in a distributed SQL engine, where incoming signals update only the relevant shard rather than recomputing the entire graph. This reduces edge resolution to seconds, enabling real-time attribution and near-instant consent propagation while keeping the lakehouse for offline analytical workloads.

    #singlestore
Aug 03•6m read time•From singlestore.com
Post cover image
Table of contents
The circularity of match ratesWhy does the batch window keep growing?What does continuous resolution change?The obsolescence of the nightly schedule
2.2K Impressions
SingleStore's image
SingleStore

SingleStore Blog offers insights, tutorials, and updates on SingleStore, a distributed SQL database ...

54 Followers

•

865 Upvotes

Would you recommend this post?

Copy link
WhatsApp
Facebook
X
New Squad
  • © 2026 Daily Dev Ltd.
  • Guidelines
  • Explore
  • Tags
  • Sources
  • Squads
  • Leaderboard