--- title: Sizing Red Hat Quay: Profiling components to find real... description: A Red Hat team profiled every component of a Red Hat Quay deployment to find real pull-throughput limits for a customer running 160 nodes across 2 edge... canonical: https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Sizing Red Hat Quay: Profiling components to find real limits | daily.dev og:description: A Red Hat team profiled every component of a Red Hat Quay deployment to find real pull-throughput limits for a customer running 160 nodes across 2 edge... og:url: https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc og:image: https://api.daily.dev/og/posts/1cfTAPaTc.png og:image:alt: Sizing Red Hat Quay: Profiling components to find real limits og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Sizing Red Hat Quay: Profiling components to find real limits **[Red Hat Developer](https://daily.dev/sources/rhdev)** · 9 min read · 0 upvotes · 0 comments ## Summary A Red Hat team profiled every component of a Red Hat Quay deployment to find real pull-throughput limits for a customer running 160 nodes across 2 edge clusters. Quadrupling Quay's vCPU from 2 to 8 barely moved throughput, because NooBaa object storage was the real bottleneck. Scaling NooBaa from 4 to 6 endpoints more than doubled throughput and fixed a 49% failure rate. A second bottleneck turned out to be the Python-based load test client itself: Python's GIL capped concurrency, and splitting 400 concurrent pulls across 4 pods gave a 21x throughput improvement. They also hit a kubelet podPidsLimit of 16,384 at 5,000 concurrency and raised it to 65,536. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why did doubling Quay's vCPU from 8 to 16 not improve pull throughput? Because the test client itself became the bottleneck, not the Quay server. At 16 vCPU, Quay served data faster than the Python test client could consume it, causing TCP zero-window packets. The root cause was Python's Global Interpreter Lock, which let only one thread execute bytecode at a time, so acquiring and releasing the lock across 400+ threads overwhelmed the actual HTTP work. _Anyone load-testing their own registry can compare notes on client-side bottlenecks via daily.dev._ ### How many NooBaa object storage endpoints do I need to avoid pull failures under load on Red Hat Quay? For an 8 vCPU Quay configuration handling 500 concurrent pulls, 4 NooBaa endpoints caused a 49% failure rate due to CPU saturation, while scaling to 6 endpoints achieved 100% success and more than doubled throughput from 0.22 to 0.52 images per second. Storage must be sized proportionally with the Quay application tier. _Teams sizing registry storage can track real-world benchmarks like this on daily.dev before committing resources._ ### What kubelet setting limits the number of concurrent threads or processes on an OpenShift node? The kubelet's podPidsLimit caps process and thread counts per pod, defaulting to 16,384 in one tested deployment. Under a 5,000-concurrency load test this limit was hit and had to be raised to 65,536 using a KubeletConfig custom resource, with matching cgroup limits set in the container. _Engineers tuning OpenShift nodes for high-concurrency workloads can keep up with limits like this on daily.dev._ ## Similar posts on daily.dev - [Kubernetes Requests and Limits: How to Right-Size Pods Without Breaking Reliability](https://daily.dev/posts/kubernetes-requests-and-limits-how-to-right-size-pods-without-breaking-reliability-2idykp32g) · Cast AI · 0 upvotes · 0 comments - [Mount Mayhem at Netflix: Scaling Containers on Modern CPUs](https://daily.dev/posts/mount-mayhem-at-netflix-scaling-containers-on-modern-cpus-ckvvan6ve) · Netflix TechBlog · 50 upvotes · 1 comments - [Egress network quality of service on Red Hat OpenShift](https://daily.dev/posts/egress-network-quality-of-service-on-red-hat-openshift-nrdopw22m) · Red Hat Developer · 0 upvotes · 0 comments - [Netflix Uncovers Kernel-Level Bottlenecks While Scaling Containers on Modern CPUs](https://daily.dev/posts/netflix-uncovers-kernel-level-bottlenecks-while-scaling-containers-on-modern-cpus-bfs33g6uj) · InfoQ · 3 upvotes · 0 comments - [How we built a 130,000-node GKE cluster](https://daily.dev/posts/how-we-built-a-130-000-node-gke-cluster-t7cc5stwb) · Hacker News · 0 upvotes · 0 comments --- Tags: [#containers](https://daily.dev/tags/containers), [#openshift](https://daily.dev/tags/openshift) [View this post on daily.dev](https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Sizing Red Hat Quay: Profiling components to find real limits","url":"https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc"},"datePublished":"2026-10-29T13:26:17.460Z","dateModified":"2026-10-29T13:26:49.940Z","description":"A Red Hat team profiled every component of a Red Hat Quay deployment to find real pull-throughput limits for a customer running 160 nodes across 2 edge...","image":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","thumbnailUrl":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","isAccessibleForFree":true,"articleSection":"Red Hat Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Red Hat Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/2a8895f3216c4900bdc455585a4ad3e4","url":"https://daily.dev/sources/rhdev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"containers,openshift","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Red Hat Developer","item":"https://daily.dev/sources/rhdev"},{"@type":"ListItem","position":3,"name":"Sizing Red Hat Quay: Profiling components to find real limits"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/sizing-red-hat-quay-profiling-components-to-find-real-limits-1cftapatc#faq","mainEntity":[{"@type":"Question","name":"Why did doubling Quay's vCPU from 8 to 16 not improve pull throughput?","acceptedAnswer":{"@type":"Answer","text":"Because the test client itself became the bottleneck, not the Quay server. At 16 vCPU, Quay served data faster than the Python test client could consume it, causing TCP zero-window packets. The root cause was Python's Global Interpreter Lock, which let only one thread execute bytecode at a time, so acquiring and releasing the lock across 400+ threads overwhelmed the actual HTTP work. Anyone load-testing their own registry can compare notes on client-side bottlenecks via daily.dev."}},{"@type":"Question","name":"How many NooBaa object storage endpoints do I need to avoid pull failures under load on Red Hat Quay?","acceptedAnswer":{"@type":"Answer","text":"For an 8 vCPU Quay configuration handling 500 concurrent pulls, 4 NooBaa endpoints caused a 49% failure rate due to CPU saturation, while scaling to 6 endpoints achieved 100% success and more than doubled throughput from 0.22 to 0.52 images per second. Storage must be sized proportionally with the Quay application tier. Teams sizing registry storage can track real-world benchmarks like this on daily.dev before committing resources."}},{"@type":"Question","name":"What kubelet setting limits the number of concurrent threads or processes on an OpenShift node?","acceptedAnswer":{"@type":"Answer","text":"The kubelet's podPidsLimit caps process and thread counts per pod, defaulting to 16,384 in one tested deployment. Under a 5,000-concurrency load test this limit was hit and had to be raised to 65,536 using a KubeletConfig custom resource, with matching cgroup limits set in the container. Engineers tuning OpenShift nodes for high-concurrency workloads can keep up with limits like this on daily.dev."}}]} ```