Daniel Stenberg describes building a new performance testing system for curl from scratch, after years of shelving the idea. Running on his own local development machine (to avoid noisy cloud infrastructure), a shell script pulls the latest git source, builds curl, and runs 24 tests every twenty minutes via crontab, logging results and generating gnuplot graphs. Results are automatically published live at curl.se/perf. The setup already surfaced regressions within hours of going live, prompting quick fixes from curl developers. New concepts introduced include 'stakes' (manually set thresholds to flag regressions), box-and-whisker plots for distribution, LTTB downsampling for long-term graphs, and the Mann-Kendall Test + Sen's Slope for trend detection. The code lives in a new curl/perf GitHub repository, and Stenberg is inviting contributions.
Table of contents
Let’s do thisLet me do thisIt’s live babyDistributionChanging conditionsStakesBalancesThe testsBuild volumes and graphsSpotting changeDevelopingQuestions this post answers
How does curl detect performance regressions in its codebase over time?
Curl runs a new performance test suite roughly every twenty minutes via crontab, building from the latest git commits on a dedicated development machine and running 24 tests that take about six minutes total. Results feed a live public dashboard at curl.se/perf using gnuplot graphs, box-and-whisker plots for distribution, and manually set thresholds called 'stakes' to flag when a merge degrades performance. daily.dev surfaces engineering writeups like this for teams building their own regression-detection pipelines.
What algorithm can I use to downsample time series data for long-term performance graphs?
Largest-Triangle-Three-Buckets (LTTB) is an algorithm for downsampling a full range of data points into a smaller set that remains visually representative on a graph. Curl's performance dashboard uses LTTB to generate separate 'long term' graphs once the number of build data points grows too large to display conveniently in a single short-term chart. developers tuning dashboards and telemetry pipelines can find similar data-visualization techniques on daily.dev.
What statistical test can detect gradual performance trends across many builds?
The Mann-Kendall Test combined with Sen's Slope is used to identify trends in noisy time-series data, such as performance measurements across hundreds of software builds. It can be applied to spot a slow, gradual degradation or improvement that individual data points wouldn't reveal, complementing manually set threshold markers for catching sudden regressions. daily.dev helps engineers tracking build performance discover trend-detection techniques like this one.
37K Impressions1 Comment