--- title: Notion scales vector search 10x at one-tenth the cost,... description: Notion cut vector search infrastructure costs by 90% through chunk hashing, metadata splitting, and a move to serverless plus Ray for unified CPU/GPU... canonical: https://daily.dev/posts/notion-scales-vector-search-10x-at-one-tenth-the-cost-litellm-rewrites-gateway-in-rust-7m7cllizj twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Notion scales vector search 10x at one-tenth the cost, LiteLLM rewrites gateway in Rust | daily.dev og:description: Notion cut vector search infrastructure costs by 90% through chunk hashing, metadata splitting, and a move to serverless plus Ray for unified CPU/GPU... og:url: https://daily.dev/posts/notion-scales-vector-search-10x-at-one-tenth-the-cost-litellm-rewrites-gateway-in-rust-7m7cllizj og:image: https://api.daily.dev/og/posts/7M7ClLiZj.png og:image:alt: Notion scales vector search 10x at one-tenth the cost, LiteLLM rewrites gateway in Rust og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Notion scales vector search 10x at one-tenth the cost, LiteLLM rewrites gateway in Rust **[Backend Digest](https://daily.dev/sources/backend_digest)** · 4 min read · 0 upvotes · 0 comments ## Summary Notion cut vector search infrastructure costs by 90% through chunk hashing, metadata splitting, and a move to serverless plus Ray for unified CPU/GPU processing. LiteLLM is migrating its AI gateway from Python to Rust, hitting 0.05ms overhead and 6,782 req/s in benchmarks — a 15x throughput jump. PostgreSQL 19 lands io_uring as a first-class io_method, enabling kernel-managed async reads with multiple in-flight I/Os. Six North Korea-linked npm packages impersonating Rollup polyfill tools were caught stealing AWS, Azure, and AI provider credentials in a Lazarus Group supply chain campaign. ## Content **TLDR:** Notion cut vector search infrastructure costs by 90% through chunk hashing, metadata splitting, and a move to serverless plus Ray for unified CPU/GPU processing. LiteLLM is migrating its AI gateway from Python to Rust, hitting 0.05ms overhead and 6,782 req/s in benchmarks — a 15x throughput jump. PostgreSQL 19 lands io_uring as a first-class io_method, enabling kernel-managed async reads with multiple in-flight I/Os. Six North Korea-linked npm packages impersonating Rollup polyfill tools were caught stealing AWS, Azure, and AI provider credentials in a Lazarus Group supply chain campaign. --- ## Notion scales vector search to 10x capacity at one-tenth the original cost The engineering involved three concrete changes: chunk hashing to skip re-embedding unchanged content on edits, splitting chunks into metadata and textual content so permission changes don't trigger full re-embedding (70% data volume reduction), and migrating to Ray for unified CPU/GPU processing on the same node (anticipated 90% embedding cost reduction). Moving to a serverless architecture alone cut costs 50%. The result is a system that handles tens of thousands of workspaces daily through Apache Spark batch jobs with Kafka handling real-time updates — worth studying if you're running your own embedding pipeline at any meaningful scale. [Read more](https://daily.dev/posts/lkqA2n49A) ## LiteLLM migrates AI gateway from Python to Rust: 15x throughput, 0.05ms overhead Benchmarks show the Rust gateway at ~0.05ms per-request overhead versus ~7.5ms for Python, 6,782 req/s versus 453, and 32MB memory versus 359MB. The migration is route-by-route — OCR first, then /messages, then /chat/completions — with a full axum server targeted by December 2026. Config, database schema, and client API stay unchanged throughout. An early beta is open if you want to test it against your workload before the full rollout. [Read more](https://daily.dev/posts/J1EVV50SF) ## PostgreSQL 19 adds io_uring as a native io_method for async buffered reads Unlike the worker method, which uses dedicated IO worker processes issuing synchronous pread64 calls, io_uring submits reads directly to the kernel's submission queue and retrieves completions asynchronously, allowing multiple reads in-flight simultaneously. The post demonstrates this via strace showing io_uring_enter syscalls and EXPLAIN ANALYZE output with prefetch and I/O statistics. One monitoring gotcha: with io_uring and many parallel workers, load average can spike above 25 while CPUs stay nearly idle — your existing alerting thresholds will lie to you. [Read more](https://daily.dev/posts/OzZBu63EC) ## Lazarus Group npm campaign targets developer credentials via fake Rollup polyfill packages JFrog researchers identified six malicious packages using a multi-stage delivery chain — hiding second-stage payloads in fake SVG utilities that fetched and executed remote JSON — to steal credentials for AWS, Azure, Google Gemini, Anthropic Claude, and SSH keys. VS Code, Windsurf, and Cursor configs were also targeted. The packages have been removed from npm, but the campaign shares traits with OtterCookie malware from the Contagious Interview operation, including a forked input-control library enabling remote terminal sessions. If your team uses any Rollup-adjacent tooling, audit your dependency tree. [Read more](https://daily.dev/posts/mvjcoUuHx) --- ## Also notable - **ClickHouse Managed Postgres runs a PgBouncer fleet via so_reuseport, hitting ~336k TPS on 16 vCPUs:** A single PgBouncer process caps at one CPU core; ClickHouse's approach runs one process per core sharing the same port via so_reuseport, with cancel requests forwarded through process peering — benchmarks on a c7i.4xlarge show ~336k TPS versus ~87k for a single process, using 52% of box CPU versus 9%. [Read more](https://daily.dev/posts/dIq3cL9Xo) - **Oracle silently halved Always Free A1 Compute limits on June 15 with no announcement:** Limits dropped from 4 OCPUs/24 GB RAM to 2 OCPUs/12 GB RAM; grandfathered PAYG instances face a hidden risk — if terminated for any reason, they cannot be recreated above the new limits, so audit your A1 instances and confirm your account status with Oracle Support directly. [Read more](https://daily.dev/posts/LDGJRD9Vs) - **Alibaba's SkillWeaver cuts agent token usage from ~884,000 to ~1,160 per query on 2,209 MCP skills:** The Skill-Aware Decomposition feedback loop rewrites task breakdowns using vocabulary aligned with actual available tools; benchmarked on 300 multi-step queries, it reduced token consumption 99.9% while improving accuracy — source code isn't released yet but the SAD technique is reproducible with LangChain or raw Python using prompt templates from the paper. [Read more](https://daily.dev/posts/tiVKPPigh) - **OpenTelemetry graduates to CNCF's highest maturity level, joining Kubernetes and Prometheus:** The graduation formally marks OTel as the vendor-neutral standard for metrics, logs, and traces at enterprise scale, with vendors expected to differentiate through AI-assisted root cause analysis built on top of the common data model rather than proprietary collection layers. [Read more](https://daily.dev/posts/58hGMhIKa) - **DSpark improves DeepSeek-V4 per-user generation speed 60-85% over MTP-1 baseline via speculative decoding:** The framework combines semi-autoregressive generation with confidence-scheduled verification to address slow autoregressive drafters and suffix decay; DeepSeek also released DeepSpec, an MIT-licensed training and evaluation framework for speculative decoding drafters. [Read more](https://daily.dev/posts/tEOGimJDI) --- Tags: [#security](https://daily.dev/tags/security), [#rust](https://daily.dev/tags/rust), [#postgresql](https://daily.dev/tags/postgresql), [#vector-search](https://daily.dev/tags/vector-search) [View this post on daily.dev](https://daily.dev/posts/notion-scales-vector-search-10x-at-one-tenth-the-cost-litellm-rewrites-gateway-in-rust-7m7cllizj) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/notion-scales-vector-search-10x-at-one-tenth-the-cost-litellm-rewrites-gateway-in-rust-7m7cllizj","headline":"Notion scales vector search 10x at one-tenth the cost, LiteLLM rewrites gateway in Rust","text":"Notion cut vector search infrastructure costs by 90% through chunk hashing, metadata splitting, and a move to serverless plus Ray for unified CPU/GPU processing. LiteLLM is migrating its AI gateway from Python to Rust, hitting 0.05ms overhead and 6,782 req/s in benchmarks — a 15x throughput jump. PostgreSQL 19 lands io_uring as a first-class io_method, enabling kernel-managed async reads with multiple in-flight I/Os. Six North Korea-linked npm packages impersonating Rollup polyfill tools were caught stealing AWS, Azure, and AI provider credentials in a Lazarus Group supply chain campaign.","url":"https://daily.dev/posts/notion-scales-vector-search-10x-at-one-tenth-the-cost-litellm-rewrites-gateway-in-rust-7m7cllizj","datePublished":"2026-07-04T04:18:45.791Z","dateModified":"2026-07-04T04:19:01.646Z","author":{"@type":"Organization","name":"Backend Digest","logo":"https://media.daily.dev/image/upload/s--9uCSrsAb--/f_auto,q_auto/v1773839398/logos/backend_digest?_a=BAMAMiiu0","url":"https://daily.dev/sources/backend_digest"},"interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/sources/backend_digest","name":"Backend Digest"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Backend Digest","item":"https://daily.dev/sources/backend_digest"},{"@type":"ListItem","position":3,"name":"Notion scales vector search 10x at one-tenth the cost, LiteLLM rewrites gateway in Rust"}]} ```