Twitter customers are affected by network latency, and an experiment was conducted to lower latency using a faster public cloud edge. The impact of the experiment was measured using the CausalImpact package from Google, which offers flexibility and modularity. A robust framework was developed for the causal impact analysis, and key metrics were selected to quantify the impact. The analysis revealed insights and challenges, including the importance of validation techniques and the potential impact of external shocks.
Table of contents
Inferring causality with Bayesian Structural Time Series (BSTS)Constructing a robust analysis frameworkEmphasis on experiment design and setupValidating the success of experiment rolloutFiltering the right metricsObservation period and challengesModel tuning and evaluationKey learnings and takeawaysResults validation and robustness are crucialZooming on counterintuitive resultsGrab opportunities from external shocksWhat’s next from here?Acknowledgements23 Impressions