--- title: Inference-time scaling on Red Hat AI: Improving model... description: Inference-time scaling (ITS) improves language model reliability by generating multiple candidate outputs at runtime and selecting the best one — without... canonical: https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Inference-time scaling on Red Hat AI: Improving model reliability | daily.dev og:description: Inference-time scaling (ITS) improves language model reliability by generating multiple candidate outputs at runtime and selecting the best one — without... og:url: https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik og:image: https://api.daily.dev/og/posts/vOr0BMlIk.png og:image:alt: Inference-time scaling on Red Hat AI: Improving model reliability og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Inference-time scaling on Red Hat AI: Improving model reliability **[Red Hat Developer](https://daily.dev/sources/rhdev)** · 9 min read · 1 upvotes · 0 comments ## Summary Inference-time scaling (ITS) improves language model reliability by generating multiple candidate outputs at runtime and selecting the best one — without retraining or fine-tuning. Red Hat AI's open source `its_hub` framework packages techniques like self-consistency, best-of-N, hierarchical voting, and particle filtering into a Python SDK and OpenAI-compatible HTTP API. Applied to Red Hat OpenShift Lightspeed's agentic tool-calling workflows, ITS improved pass rates from 10% to 20% across 11 troubleshooting scenarios, with the largest gains on the hardest multi-step tasks. The framework works with any OpenAI-compatible endpoint including vLLM, and Red Hat plans to integrate ITS into its AI gateway as an infrastructure-level capability. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is its_hub and how does it improve model reliability at inference time? its_hub is an open source framework from Red Hat AI that applies inference-time scaling techniques such as best-of-N, self-consistency, hierarchical voting, particle filtering, and beam search to improve model output reliability without retraining or fine-tuning. It works via a Python SDK compatible with any OpenAI-compatible endpoint (including vLLM) or as an inference-as-a-service HTTP API, and supports reward-hub integration for custom scoring. _Teams evaluating ways to make agentic tool calling more reliable can track frameworks like its_hub on daily.dev._ ### How much did inference-time scaling improve tool-calling reliability in the OpenShift Lightspeed experiment? Applying hierarchical voting with its_hub to Red Hat OpenShift Lightspeed improved overall pass rates from 10% to 20% across 11 troubleshooting scenarios. The biggest gains occurred on the hardest scenarios, where multi-step log analysis tasks went from near-zero success to consistent resolution, while already-reliable scenarios stayed reliable. _Anyone benchmarking agent reliability gains can follow real-world results like this one on daily.dev._ ### What is hierarchical voting in inference-time scaling and when should I use it? Hierarchical voting extends self-consistency to tool-calling workflows by sampling multiple candidates in parallel and voting in two stages: first on which tool to call, then on the arguments to pass. It is designed for agentic tasks where the critical decision is choosing the correct action rather than generating text, making it effective at reducing cascading failures from a single wrong tool call. _Developers designing agent tool-selection logic can compare techniques like this via daily.dev._ ## Similar posts on daily.dev - [AI quickstart: Self-service agent for IT process automation](https://daily.dev/posts/ai-quickstart-self-service-agent-for-it-process-automation-onck4fj2w) · Red Hat Developer · 1 upvotes · 0 comments - [Red Hat launches unified platform for deploying and managing AI models, agents, and apps](https://daily.dev/posts/red-hat-launches-unified-platform-for-deploying-and-managing-ai-models-agents-and-apps-e1iaecils) · SD Times · 1 upvotes · 0 comments - [Red Hat’s skill packs give AI agents something a bigger model never could: 20 years of institutional memory](https://daily.dev/posts/red-hat-s-skill-packs-give-ai-agents-something-a-bigger-model-never-could-20-years-of-institutional-onyqocvjh) · The New Stack · 0 upvotes · 0 comments - [Red Hat ships AI platform for hybrid cloud deployments](https://daily.dev/posts/red-hat-ships-ai-platform-for-hybrid-cloud-deployments-wkafxoetq) · InfoWorld · 0 upvotes · 0 comments - [IBM Research uses vLLM at the heart of its RITS Platform – PyTorch](https://daily.dev/posts/ibm-research-uses-vllm-at-the-heart-of-its-rits-platform-pytorch-umwekupnu) · PyTorch · 0 upvotes · 0 comments --- Tags: [#agentic-ai](https://daily.dev/tags/agentic-ai) [View this post on daily.dev](https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Inference-time scaling on Red Hat AI: Improving model reliability","url":"https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik"},"datePublished":"2026-07-31T13:42:30.302Z","dateModified":"2026-09-13T21:32:41.181Z","description":"Inference-time scaling (ITS) improves language model reliability by generating multiple candidate outputs at runtime and selecting the best one — without...","image":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","thumbnailUrl":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","isAccessibleForFree":true,"articleSection":"Red Hat Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Red Hat Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/2a8895f3216c4900bdc455585a4ad3e4","url":"https://daily.dev/sources/rhdev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"agentic-ai","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Red Hat Developer","item":"https://daily.dev/sources/rhdev"},{"@type":"ListItem","position":3,"name":"Inference-time scaling on Red Hat AI: Improving model reliability"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/inference-time-scaling-on-red-hat-ai-improving-model-reliability-vor0bmlik#faq","mainEntity":[{"@type":"Question","name":"What is its_hub and how does it improve model reliability at inference time?","acceptedAnswer":{"@type":"Answer","text":"its_hub is an open source framework from Red Hat AI that applies inference-time scaling techniques such as best-of-N, self-consistency, hierarchical voting, particle filtering, and beam search to improve model output reliability without retraining or fine-tuning. It works via a Python SDK compatible with any OpenAI-compatible endpoint (including vLLM) or as an inference-as-a-service HTTP API, and supports reward-hub integration for custom scoring. Teams evaluating ways to make agentic tool calling more reliable can track frameworks like its_hub on daily.dev."}},{"@type":"Question","name":"How much did inference-time scaling improve tool-calling reliability in the OpenShift Lightspeed experiment?","acceptedAnswer":{"@type":"Answer","text":"Applying hierarchical voting with its_hub to Red Hat OpenShift Lightspeed improved overall pass rates from 10% to 20% across 11 troubleshooting scenarios. The biggest gains occurred on the hardest scenarios, where multi-step log analysis tasks went from near-zero success to consistent resolution, while already-reliable scenarios stayed reliable. Anyone benchmarking agent reliability gains can follow real-world results like this one on daily.dev."}},{"@type":"Question","name":"What is hierarchical voting in inference-time scaling and when should I use it?","acceptedAnswer":{"@type":"Answer","text":"Hierarchical voting extends self-consistency to tool-calling workflows by sampling multiple candidates in parallel and voting in two stages: first on which tool to call, then on the arguments to pass. It is designed for agentic tasks where the critical decision is choosing the correct action rather than generating text, making it effective at reducing cascading failures from a single wrong tool call. Developers designing agent tool-selection logic can compare techniques like this via daily.dev."}}]} ```