--- title: What building Shippy taught us about building agents description: The Skylight team at Ai2 shares the architecture and lessons learned from building Shippy, a maritime AI agent for real-time ocean domain awareness. The post... canonical: https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What building Shippy taught us about building agents | daily.dev og:description: The Skylight team at Ai2 shares the architecture and lessons learned from building Shippy, a maritime AI agent for real-time ocean domain awareness. The post... og:url: https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx og:image: https://api.daily.dev/og/posts/DGVBCXdcX.png og:image:alt: What building Shippy taught us about building agents og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What building Shippy taught us about building agents **[Hugging Face](https://daily.dev/sources/huggingface)** · 10 min read · 2 upvotes · 0 comments ## Summary The Skylight team at Ai2 shares the architecture and lessons learned from building Shippy, a maritime AI agent for real-time ocean domain awareness. The post covers their three-part agent model (soul, skills, config), the use of a purpose-built CLI to wrap a complex API and reduce nondeterministic errors, a Kubernetes-based sandboxed hosting platform called Mothership that isolates each user session, and a custom evaluation framework using subject-matter expert rubrics and LLM judges scored against live data. Key lessons include wrapping complex APIs in deterministic CLIs, making behavioral guardrails explicit in system prompts rather than fine-tuning, and evaluating the full agent stack against real data rather than static benchmarks. Upcoming work includes agent-driven UI control, model routing, and cross-thread memory. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do you prevent an AI agent from making malformed or buggy API calls when interacting with a complex API? Wrap the API in a deterministic, purpose-built CLI rather than letting the agent construct raw calls. In Shippy's case, early prototypes that let the agent build API calls directly produced malformed pagination, geometry encoding errors, and misunderstood filter types. A CLI with typed filter flags, self-documenting help text, and file-based output (instead of piping through the shell, which hit buffer limits) collapsed that complexity into a predictable interface. _daily.dev surfaces practical patterns like this CLI-wrapping approach for developers building reliable agent tool interfaces._ ### How do you isolate user sessions when building a multi-tenant AI agent platform? Provision a dedicated, ephemeral Kubernetes deployment per user session rather than sharing infrastructure across users. Ai2's Mothership platform spins up pods packaging the agent runtime, its skills, and a CLI for each session, injecting the user's JWT at provision time so API calls are scoped to that user's data, and session files are never shared across users. _Developers designing multi-tenant agent infrastructure can track approaches like this one on daily.dev._ ### How do you evaluate an AI agent instead of just the underlying model? Score the whole agent stack together against live data, not static benchmark questions. Ai2 built a custom eval system using the Harbor framework where subject-matter experts write scenarios and weighted rubrics per task, an LLM judge grades each criterion 0 to 1 with written reasoning, and a weighted aggregate is checked against a fixed pass threshold before any version reaches users. _daily.dev helps teams designing agent evaluation pipelines keep up with real-world approaches like this one._ ## Similar posts on daily.dev - [AI Agent Fleet for CI/CD: How Docker Ships Faster](https://daily.dev/posts/ai-agent-fleet-for-ci-cd-how-docker-ships-faster-acrgrdbdx) · Docker · 29 upvotes · 0 comments - [One Agent, One Machine](https://daily.dev/posts/one-agent-one-machine-ypl6zokth) · ElixirStatus · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#kubernetes](https://daily.dev/tags/kubernetes), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What building Shippy taught us about building agents","url":"https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx"},"datePublished":"2026-07-15T17:29:48.707Z","dateModified":"2026-09-13T21:30:09.828Z","description":"The Skylight team at Ai2 shares the architecture and lessons learned from building Shippy, a maritime AI agent for real-time ocean domain awareness. The post...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ce6bb1860b26aee46af8c805ecc1a512?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ce6bb1860b26aee46af8c805ecc1a512?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hugging Face","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hugging Face","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/f1f55c67d81a4330acf5b90b26b0c8e1","url":"https://daily.dev/sources/huggingface"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,kubernetes,ai-agents","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hugging Face","item":"https://daily.dev/sources/huggingface"},{"@type":"ListItem","position":3,"name":"What building Shippy taught us about building agents"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/what-building-shippy-taught-us-about-building-agents-dgvbcxdcx#faq","mainEntity":[{"@type":"Question","name":"How do you prevent an AI agent from making malformed or buggy API calls when interacting with a complex API?","acceptedAnswer":{"@type":"Answer","text":"Wrap the API in a deterministic, purpose-built CLI rather than letting the agent construct raw calls. In Shippy's case, early prototypes that let the agent build API calls directly produced malformed pagination, geometry encoding errors, and misunderstood filter types. A CLI with typed filter flags, self-documenting help text, and file-based output (instead of piping through the shell, which hit buffer limits) collapsed that complexity into a predictable interface. daily.dev surfaces practical patterns like this CLI-wrapping approach for developers building reliable agent tool interfaces."}},{"@type":"Question","name":"How do you isolate user sessions when building a multi-tenant AI agent platform?","acceptedAnswer":{"@type":"Answer","text":"Provision a dedicated, ephemeral Kubernetes deployment per user session rather than sharing infrastructure across users. Ai2's Mothership platform spins up pods packaging the agent runtime, its skills, and a CLI for each session, injecting the user's JWT at provision time so API calls are scoped to that user's data, and session files are never shared across users. Developers designing multi-tenant agent infrastructure can track approaches like this one on daily.dev."}},{"@type":"Question","name":"How do you evaluate an AI agent instead of just the underlying model?","acceptedAnswer":{"@type":"Answer","text":"Score the whole agent stack together against live data, not static benchmark questions. Ai2 built a custom eval system using the Harbor framework where subject-matter experts write scenarios and weighted rubrics per task, an LLM judge grades each criterion 0 to 1 with written reasoning, and a weighted aggregate is checked against a fixed pass threshold before any version reaches users. daily.dev helps teams designing agent evaluation pipelines keep up with real-world approaches like this one."}}]} ```