--- title: Polymorphic Inference: self-tuning inference for open models description: Together AI introduces 'Polymorphic Inference,' a framework for adapting inference systems to constantly changing models, traffic, and hardware, rather than... canonical: https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Polymorphic Inference: self-tuning inference for open models | daily.dev og:description: Together AI introduces 'Polymorphic Inference,' a framework for adapting inference systems to constantly changing models, traffic, and hardware, rather than... og:url: https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces og:image: https://api.daily.dev/og/posts/QcpxdSCeS.png og:image:alt: Polymorphic Inference: self-tuning inference for open models og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Polymorphic Inference: self-tuning inference for open models **[Together AI](https://daily.dev/sources/togetherai)** · 9 min read · 0 upvotes · 0 comments ## Summary Together AI introduces 'Polymorphic Inference,' a framework for adapting inference systems to constantly changing models, traffic, and hardware, rather than tuning for a single static configuration. The company announces Together Compiler, which turns inference optimization into an agentic search problem using swarms of AI agents instead of hand-written kernels. Together Compiler is already running in production on popular serverless endpoints and is available to select customers in limited beta. The post cites that 80% of OpenRouter traffic goes to models released in the last three months, and that the top ten OSS models all use different architectures. Upcoming work includes self-serve Continuous Compilation, shape-shifting speculative decoding, and a ThunderAgent router for heterogeneous deployments. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Together Compiler and how does it optimize LLM inference? Together Compiler is an AI-powered performance optimization system from Together AI that recasts inference tuning as an agentic search problem rather than a traditional compiler translation task. It pairs a swarm of agents with a high-fidelity persistent engine for fast iteration, identifying key optimization patterns automatically. It already runs in production on Together AI's popular serverless endpoints, finding further gains on top of already optimized code, and is in limited beta for customers. _Teams weighing agentic versus hand-tuned inference optimization can track approaches like this on daily.dev._ ### Why is a static vLLM or SGLang configuration not optimal for all production traffic? A hyperparameter-tuned configuration optimized for one setting is not necessarily best for another, because model architectures, traffic shapes, and hardware keep changing. For example, an MoE kernel tuned against random tokens may not perform well on real production traffic patterns, since open-source kernels are often released assuming static traffic shapes that don't match actual workloads. _Engineers tuning vLLM or SGLang deployments can follow this kind of inference-tuning tradeoff on daily.dev._ ### What percentage of OpenRouter traffic goes to recently released open-source models? 80% of traffic on OpenRouter goes to models released in the last three months, reflecting the rapid pace of new open-source model releases. This is cited as evidence of the fast-changing model landscape, alongside the observation that the ten most popular open-source models in use all use different architectures from one another. _Developers tracking open-source model adoption trends can follow data points like this on daily.dev._ ## Similar posts on daily.dev - [Together Compiler: Continuous Compilation for AI inference](https://daily.dev/posts/together-compiler-continuous-compilation-for-ai-inference-bcppprcsu) · Together AI · 1 upvotes · 0 comments - [OpenCode Now Supports DigitalOcean Inference Router for Intelligent Model Routing](https://daily.dev/posts/opencode-now-supports-digitalocean-inference-router-for-intelligent-model-routing-q5jl2xpmo) · DigitalOcean · 0 upvotes · 0 comments - [State of AI](https://daily.dev/posts/state-of-ai-dlul2m2pu) · Hacker News · 3 upvotes · 0 comments - [The production platform for open-weight AI inference](https://daily.dev/posts/the-production-platform-for-open-weight-ai-inference-rcdyvkjtr) · Together AI · 0 upvotes · 0 comments - [Best OpenAI-compatible inference APIs: drop-in alternatives for 2026](https://daily.dev/posts/best-openai-compatible-inference-apis-drop-in-alternatives-for-2026-vrtnzttyy) · DigitalOcean Community · 1 upvotes · 0 comments --- Tags: [#open-source](https://daily.dev/tags/open-source), [#ai-agents](https://daily.dev/tags/ai-agents), [#ai-inference](https://daily.dev/tags/ai-inference), [#together-ai](https://daily.dev/tags/together-ai) [View this post on daily.dev](https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Polymorphic Inference: self-tuning inference for open models","url":"https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces"},"datePublished":"2026-11-04T14:01:25.594Z","dateModified":"2026-11-04T14:02:06.746Z","description":"Together AI introduces 'Polymorphic Inference,' a framework for adapting inference systems to constantly changing models, traffic, and hardware, rather than...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/74a2027504bcc8e13efed92552e95f0d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/74a2027504bcc8e13efed92552e95f0d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Together AI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Together AI","logo":"https://media.daily.dev/image/upload/s--tCjWcJfJ--/f_auto,q_auto/v1780213200/logos/togetherai?_a=BAMAMiWQ0","url":"https://daily.dev/sources/togetherai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"open-source,ai-agents,ai-inference,together-ai","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Together AI","item":"https://daily.dev/sources/togetherai"},{"@type":"ListItem","position":3,"name":"Polymorphic Inference: self-tuning inference for open models"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/polymorphic-inference-self-tuning-inference-for-open-models-qcpxdsces#faq","mainEntity":[{"@type":"Question","name":"What is Together Compiler and how does it optimize LLM inference?","acceptedAnswer":{"@type":"Answer","text":"Together Compiler is an AI-powered performance optimization system from Together AI that recasts inference tuning as an agentic search problem rather than a traditional compiler translation task. It pairs a swarm of agents with a high-fidelity persistent engine for fast iteration, identifying key optimization patterns automatically. It already runs in production on Together AI's popular serverless endpoints, finding further gains on top of already optimized code, and is in limited beta for customers. Teams weighing agentic versus hand-tuned inference optimization can track approaches like this on daily.dev."}},{"@type":"Question","name":"Why is a static vLLM or SGLang configuration not optimal for all production traffic?","acceptedAnswer":{"@type":"Answer","text":"A hyperparameter-tuned configuration optimized for one setting is not necessarily best for another, because model architectures, traffic shapes, and hardware keep changing. For example, an MoE kernel tuned against random tokens may not perform well on real production traffic patterns, since open-source kernels are often released assuming static traffic shapes that don't match actual workloads. Engineers tuning vLLM or SGLang deployments can follow this kind of inference-tuning tradeoff on daily.dev."}},{"@type":"Question","name":"What percentage of OpenRouter traffic goes to recently released open-source models?","acceptedAnswer":{"@type":"Answer","text":"80% of traffic on OpenRouter goes to models released in the last three months, reflecting the rapid pace of new open-source model releases. This is cited as evidence of the fast-changing model landscape, alongside the observation that the ten most popular open-source models in use all use different architectures from one another. Developers tracking open-source model adoption trends can follow data points like this on daily.dev."}}]} ```