Change data capture (CDC) is presented as the fix for stale context feeding AI agents, contrasted with batch ETL pipelines that inherently lag by hours. The piece explains log-based, trigger-based, and query-based CDC approaches, cites production examples from Uber, Netflix, and LinkedIn showing freshness improvements, and references real incidents (Air Canada chatbot, Cursor support bot, a dealership chatbot) where stale or unvalidated context caused liability. It then positions Redis and its Redis Data Integration (RDI) product—a Debezium-based CDC pipeline into Redis—as the serving layer that keeps agent context synchronized with source databases within seconds.
Table of contents
What change data capture is & how it worksYour agents aren't the problemBatch ETL is stale by designAgents act on what they retrieveHow CDC & event-driven sync close the gapGrade your context layerWhere Redis fits: CDC into the store your agents readFresh context is an architecture decisionQuestions this post answers
What is the difference between log-based, trigger-based, and query-based change data capture?
Log-based CDC reads a database's transaction log directly, capturing inserts, updates, and deletes asynchronously without touching the write path, and is used by tools like Debezium with millisecond-range capture delay. Trigger-based CDC uses database triggers to write changes into a shadow table but adds latency to every write since it runs inside the source transaction. Query-based (polling) CDC repeatedly queries for new timestamps or version numbers, is simple but typically misses deletes and loses intermediate changes after downtime. daily.dev surfaces engineering deep dives like this for teams evaluating CDC architectures for their data pipelines.
Why did Air Canada get held liable for its chatbot giving wrong bereavement fare information?
A British Columbia tribunal ruled in February 2024 that Air Canada was liable for negligent misrepresentation after its support chatbot told a customer he could apply for a bereavement fare refund retroactively, which was not actually allowed under the airline's policy. The tribunal ordered Air Canada to pay 812.02 Canadian dollars and rejected the argument that the chatbot was a separate legal entity responsible for its own actions. Developers building customer-facing AI agents follow cases like this on daily.dev to gauge liability risk from stale or wrong outputs.
How much can log-based change data capture reduce data staleness compared to batch ETL pipelines?
Log-based CDC can shrink staleness from hours to seconds or even milliseconds, compared to batch ETL where a nightly or hourly job only reflects data as of its last scheduled run. LinkedIn's Databus reported end-to-end latencies in milliseconds while handling thousands of change events per second per server, and Uber's move from batch to Flink-based streaming cut data lake freshness from hours to minutes. Teams weighing CDC versus batch ETL for real-time systems track benchmarks like these on daily.dev.