--- title: "The 2.5 GB Wall: Why My tshark Wrapper Died, and How Streaming Fixed It" url: https://daily.dev/posts/the-2-5-gb-wall-why-my-tshark-wrapper-died-and-how-streaming-fixed-it-ouhaeyvj0 source_url: https://robinhayer.dev/the-2-5-gb-wall type: article source: "Awesome Go" published: 2026-08-13T20:23:40.868Z updated: 2026-08-13T20:47:45.317Z tags: ["performance", "golang", "networking"] reading_time: 4 upvotes: 3 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The 2.5 GB Wall: Why My tshark Wrapper Died, and How Streaming Fixed It **[Awesome Go](https://daily.dev/sources/awego)** · 4 min read · 3 upvotes · 0 comments ## Summary A developer describes debugging a tshark-based PCAP analysis CLI tool that ground to 6-7 hours and crashed with out-of-memory errors on a 2.5 GB capture file (1.9 million packets). The root causes were three separate full passes over the file, tshark's single-threaded nature, and full JSON dissection held entirely in memory by both tshark and the Go program. Consolidating the three queries into one pass cut runtime to 1-2 hours, and piping tshark's stdout directly into the Go program (streaming, with narrowed -T fields/-T ek output instead of full JSON) eliminated OOM crashes and dropped processing to about 70 minutes. Throughput remained single-threaded and unsolved, setting up a follow-up post on concurrency. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I avoid out-of-memory crashes when using tshark to analyze large PCAP files in Go? Stream tshark's output instead of buffering it. Pipe tshark's stdout directly into a Go program's stdin using narrowed output flags like -T fields or -T ek instead of full JSON dissection, so packets are processed line by line as they arrive rather than held entirely in memory. On a 2.5 GB file with 1.9 million packets, this eliminated OOM crashes and cut processing from 6-7 hours to about 70 minutes. _Anyone piping large PCAP captures through tshark can compare streaming approaches like this on daily.dev._ ### Why does running multiple tshark queries against the same PCAP file cause slow processing? Each separate query forces a full pass over the entire file, so three queries mean three full traversals of the capture. Consolidating three queries answering analytics, rows, and full dissection needs into a single optimized query that returns everything in one pass reduced processing time on a 2.5 GB file from 6-7 hours down to 1-2 hours, before any streaming changes were made. _Developers tuning packet-analysis pipelines can track techniques like query consolidation on daily.dev._ ## Similar posts on daily.dev - [I stopped using Wireshark for first-line troubleshooting after discovering Windows' built-in packet capture tool](https://daily.dev/posts/i-stopped-using-wireshark-for-first-line-troubleshooting-after-discovering-windows-built-in-packet--x4ki42qwx) · XDA Developers · 0 upvotes · 0 comments - [vignesh07/babyshark: Flows-first PCAP TUI \(case files, gorgeous UX\). Do do do do.](https://daily.dev/posts/vignesh07-babyshark-flows-first-pcap-tui-case-files-gorgeous-ux-do-do-do-do--v12mxt3cf) · Hacker News · 0 upvotes · 0 comments --- Tags: [#performance](https://daily.dev/tags/performance), [#golang](https://daily.dev/tags/golang), [#networking](https://daily.dev/tags/networking) [View this post on daily.dev](https://daily.dev/posts/the-2-5-gb-wall-why-my-tshark-wrapper-died-and-how-streaming-fixed-it-ouhaeyvj0)