Building scalable and efficient machine learning pipelines often requires overcoming bottlenecks in data transfer, feature retrieval, and orchestration. This talk showcases how Go was leveraged to operationalize an ML model service, transforming it into a high-performance, real-time system. By utilizing Go’s powerful concurrency model, shared memory for inter-process communication, and efficient queuing mechanisms, we reduced inference times from hours to just 10-15 minutes.

We’ll explore the architecture that decouples Go’s operational responsibilities—such as feature retrieval, queuing, and shared memory management—from the ML model’s ranking tasks. Attendees will learn how shared memory was used to transfer millions of scores efficiently, how Go’s interfaces enabled rapid prototyping, and how its strong typing and safety ensured robust system design. Real-world benchmarks, implementation details, and lessons learned will provide actionable insights for engineers tackling similar challenges in high-performance computing and distributed systems.

Whether you’re a software engineer, system architect, or distributed systems enthusiast, this talk will demonstrate how Go can be the backbone of scalable and efficient ML inference pipelines.

GopherCon (Gopher Academy)

Engineers at the New York Times ML platform team share how they used Go to operationalize a Python-based ML ranking model for targeted email campaigns at scale. The core challenge was bridging a Go-based platform with a Python ML model to generate 40M+ user-content recommendations. The solution uses interprocess communication via shared memory (shims), where a Go service packs tensor data into shared memory regions that a Python inference server (NVIDIA Triton) reads directly. Go handles initialization, batching tensors by type, checksum validation, worker pools, and goroutines for throughput. The result: cohort generation time dropped from hours to minutes, with statistically significant gains in user engagement.

GopherCon 2025: Supercharging ML Pipelines with Go - Vaidehi Thete