--- title: "Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU" url: https://daily.dev/posts/enterprise-grade-precision-for-long-context-multimodal-embedding-inference-on-cloud-tpu-vjvxajuwb source_url: https://developers.googleblog.com/enterprise-grade-precision-for-long-context-multimodal-embedding-inference-on-cloud-tpu type: article source: "Google Developers" published: 2026-08-26T15:05:08.837Z updated: 2026-08-26T15:52:26.800Z tags: ["gcp", "embeddings", "vllm", "gke"] reading_time: 5 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU **[Google Developers](https://daily.dev/sources/googledevs)** · 5 min read · 0 upvotes · 0 comments ## Summary Google Cloud engineers detail how they brought native TPU support to vLLM for serving the Qwen3 Embedding model family, targeting long-context text and multimodal embedding workloads. Key optimizations include hardware-safe vocabulary padding for tensor parallelism, attribute promotion for lazy weight loading, sharding-aware JIT pre-warming, and a hybrid StepPool mechanism to handle chunked prefill in pooling layers. The post includes a code sample for initializing Qwen3-Embedding-8B via vLLM's pooling runner, cosine-similarity parity thresholds (≥0.999 text, ≥0.995 multimodal) for validating TPU output against XPU references, and links to open-sourced GKE deployment recipes on GitHub. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I initialize Qwen3-Embedding-8B with vLLM on a TPU? Use vLLM's pooling runner with the model set to Qwen/Qwen3-Embedding-8B, runner set to 'pooling', tensor_parallel_size set to 2 for sharding across the TPU topology mesh, max_model_len of 16384, max_num_batched_tokens of 512, dtype bfloat16, and trust_remote_code enabled. Call llm.embed(prompts) to extract dense vector embeddings from the output. _daily.dev surfaces deployment recipes like this for teams scaling embedding inference on TPUs._ ### What cosine similarity threshold indicates acceptable numerical parity between TPU and XPU embedding outputs? A cosine similarity of at least 0.999 for text embeddings and at least 0.995 for multimodal embeddings is considered a passing quality threshold when comparing TPU-generated vectors against XPU golden reference vectors. Scores approaching 1.0 confirm that hardware-specific optimizations preserve mathematical parity without sacrificing accuracy. _developers validating cross-hardware embedding accuracy can track parity benchmarks like these on daily.dev._ ### Why does chunked prefill in pooling layers risk state loss on long-context embedding inference? Ultra-long contexts require chunked prefill to avoid exhausting high bandwidth memory, but splitting inference into steps risks losing pooling state across step boundaries. A hybrid StepPool approach that migrates metadata into CachedRequestState ensures pooling states accumulate correctly across steps and survive request preemptions. _daily.dev helps engineers keep up with long-context inference techniques like chunked pooling._ ## Similar posts on daily.dev - [Systems Engineering Playbook: Optimizing Qwen 3.5-397B MoE on Ironwood \(TPU7x\)](https://daily.dev/posts/systems-engineering-playbook-optimizing-qwen-3-5-397b-moe-on-ironwood-tpu7x--quc6gr0fd) · Google Developers · 1 upvotes · 0 comments --- Tags: [#gcp](https://daily.dev/tags/gcp), [#embeddings](https://daily.dev/tags/embeddings), [#vllm](https://daily.dev/tags/vllm), [#gke](https://daily.dev/tags/gke) [View this post on daily.dev](https://daily.dev/posts/enterprise-grade-precision-for-long-context-multimodal-embedding-inference-on-cloud-tpu-vjvxajuwb)