---
title: "Scaling Security Insights: how we achieved a 10x increase in global scanning capacity"
url: https://daily.dev/posts/scaling-security-insights-how-we-achieved-a-10x-increase-in-global-scanning-capacity-a8x05y84j
source_url: https://blog.cloudflare.com/scaling-security-scans
type: article
source: "Cloudflare"
published: 2026-06-12T13:49:48.074Z
updated: 2026-06-12T16:19:54.077Z
tags: ["golang", "postgresql", "distributed-systems", "apache-kafka"]
reading_time: 11
upvotes: 10
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scaling Security Insights: how we achieved a 10x increase in global scanning capacity

**[Cloudflare](https://daily.dev/sources/cloudflare)** · 11 min read · 10 upvotes · 0 comments

## Summary

Cloudflare's Security Insights team scaled their scanning throughput from 10 to over 120 scans per second without adding hardware. Key optimizations included: switching Kafka consumers to batch processing with goroutines, splitting consumers into 'fast lane' and 'slow lane' to prevent slow messages from blocking fast ones, replacing per-row Postgres inserts with a hybrid UNNEST/COPY bulk insert strategy, switching the internal API from active-active to active-passive to eliminate cross-continental latency, and redesigning the scheduler with independent zone scheduling, randomized scan times, and adaptive rate limiting. These changes enabled automatic scanning for all free accounts and doubled scanning frequency across all tiers.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://blog.cloudflare.com/scaling-security-scans>

## Similar posts on daily.dev

- [Cloudflare Processes 10M\+ Daily Insights with New Security Overview Dashboard](https://daily.dev/posts/cloudflare-processes-10m-daily-insights-with-new-security-overview-dashboard-2asrbn7n3) · InfoQ · 1 upvotes · 0 comments
- [Building a security overview dashboard for actionable insights](https://daily.dev/posts/building-a-security-overview-dashboard-for-actionable-insights-7lev43wwz) · Cloudflare · 0 upvotes · 0 comments
- [ShareChat hit a billion features per second, then it had to make it 10x cheaper](https://daily.dev/posts/sharechat-hit-a-billion-features-per-second-then-it-had-to-make-it-10x-cheaper-ezqgehrie) · The New Stack · 0 upvotes · 0 comments
- [Data at Cloudflare scale: some insights on measurement for 1,111 interns](https://daily.dev/posts/data-at-cloudflare-scale-some-insights-on-measurement-for-1-111-interns-t3yokbwcu) · Cloudflare · 1 upvotes · 0 comments

---

Tags: [#golang](https://daily.dev/tags/golang), [#postgresql](https://daily.dev/tags/postgresql), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#apache-kafka](https://daily.dev/tags/apache-kafka)

[View this post on daily.dev](https://daily.dev/posts/scaling-security-insights-how-we-achieved-a-10x-increase-in-global-scanning-capacity-a8x05y84j)