--- title: Harbor x LangChain: A Unified Stack for Evaluating Agents description: Harbor is an eval harness for running long-running, stateful agents in isolated, reproducible environments. LangChain integrates with Harbor in three ways:... canonical: https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Harbor x LangChain: A Unified Stack for Evaluating Agents | daily.dev og:description: Harbor is an eval harness for running long-running, stateful agents in isolated, reproducible environments. LangChain integrates with Harbor in three ways:... og:url: https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt og:image: https://api.daily.dev/og/posts/zIPdp0Byt.png og:image:alt: Harbor x LangChain: A Unified Stack for Evaluating Agents og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Harbor x LangChain: A Unified Stack for Evaluating Agents **[LangChain](https://daily.dev/sources/langchain)** · 8 min read · 1 upvotes · 1 comments ## Summary Harbor is an eval harness for running long-running, stateful agents in isolated, reproducible environments. LangChain integrates with Harbor in three ways: Deep Agents can run inside Harbor's sandboxed environment via a langgraph.json registry and make_graph factory; LangSmith Sandboxes provide clean cloud execution environments for each trial, enabling horizontal scaling with hundreds of parallel runs; and LangSmith Observability logs every job as a dataset and experiment with agent traces attached. Together these form a complete evaluation stack where Harbor orchestrates trials, Deep Agents builds the agents under test, LangSmith Sandboxes provide isolated execution, and LangSmith serves as the system of record for datasets, experiments, traces, and scores. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I connect a custom LangGraph or Deep Agent to the Harbor eval harness? Use Harbor's built-in langgraph agent, selected with --agent langgraph, and define a langgraph.json file that lists dependencies and maps a graph name to a make_graph factory function in agent.py. Harbor copies the agent into each trial's sandbox, installs the listed dependencies into a fresh virtual environment, and calls make_graph to build and run the compiled graph, keeping the integration to a single entry point. _Teams wiring agents into eval pipelines can track integration guides like this one on daily.dev._ ### How can I make my Harbor agent factory function model-agnostic instead of hardcoding a model? Define make_graph to accept a config argument and read config["configurable"]["model"] instead of hardcoding a model string. Harbor places whatever model is passed via the --model command-line flag into configurable.model, so the same factory function works across different models without code changes, as shown with create_deep_agent(model=config["configurable"]["model"]). _daily.dev helps developers keep up with practical patterns for building model-agnostic agent code._ ### What does the harbor-langsmith plugin do when running agent evaluations? Enabled with the --plugin langsmith flag, it records every Harbor job to LangSmith by syncing the dataset, creating an experiment, and logging a run per trial with the verifier's reward as feedback. If the agent under test supports LangSmith tracing, full step-by-step traces attach directly to the experiment; if not, the dataset, experiment, results, and feedback are still logged. _Engineers evaluating agent behavior beyond pass or fail scores can follow tooling updates like this via daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@kenerator** · 0 upvotes > This is the direction I like seeing eval stacks move toward: not just “score the final answer,” but run the agent in a reproducible environment, keep the trace as a first-class artifact, and make the dataset/experiment record durable enough to compare future changes. > > One extra layer I’d want beside judge/scorer outputs is deterministic checks: schema contracts, required/forbidden tool paths, cost ceilings, and domain rules that do not need another LLM opinion. > > I’m affiliated with [nxus.SYSTEMS](http://nxus.SYSTEMS); we’ve been publishing runnable nxusKit SDK examples around hybrid... ## Similar posts on daily.dev - [How we built an Evaluation Workflow for a Deep Agent with Harbor and LangSmith](https://daily.dev/posts/how-we-built-an-evaluation-workflow-for-a-deep-agent-with-harbor-and-langsmith-mk8llzmhj) · LangChain · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#langgraph](https://daily.dev/tags/langgraph), [#langsmith](https://daily.dev/tags/langsmith) [View this post on daily.dev](https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Harbor x LangChain: A Unified Stack for Evaluating Agents","url":"https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt"},"datePublished":"2026-07-08T19:23:02.287Z","dateModified":"2026-09-14T08:43:36.633Z","description":"Harbor is an eval harness for running long-running, stateful agents in isolated, reproducible environments. LangChain integrates with Harbor in three ways:...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6c7147c5bab1ed4e4a25ad6e3c17d3c6?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6c7147c5bab1ed4e4a25ad6e3c17d3c6?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"LangChain","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"LangChain","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/0f4f2629e0724a849823f8cd0d913e13","url":"https://daily.dev/sources/langchain"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"ai-agents,langgraph,langsmith","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"LangChain","item":"https://daily.dev/sources/langchain"},{"@type":"ListItem","position":3,"name":"Harbor x LangChain: A Unified Stack for Evaluating Agents"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt","comment":[{"@type":"Comment","text":"This is the direction I like seeing eval stacks move toward: not just “score the final answer,” but run the agent in a reproducible environment, keep the trace as a first-class artifact, and make the dataset/experiment record durable enough to compare future changes.\nOne extra layer I’d want beside judge/scorer outputs is deterministic checks: schema contracts, required/forbidden tool paths, cost ceilings, and domain rules that do not need another LLM opinion.\nI’m affiliated with nxus.SYSTEMS; we’ve been publishing runnable nxusKit SDK examples around hybrid eval/control patterns here: https://docs.nxus.systems/nxuskit/examples/","datePublished":"2026-07-08T23:02:04.509Z","url":"https://daily.dev/posts/zIPdp0Byt#c-D5nIwJMWi","author":{"@type":"Person","name":"Ken","url":"https://daily.dev/kenerator","image":"https://avatars.githubusercontent.com/u/95498894?v=4"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/harbor-x-langchain-a-unified-stack-for-evaluating-agents-zipdp0byt#faq","mainEntity":[{"@type":"Question","name":"How do I connect a custom LangGraph or Deep Agent to the Harbor eval harness?","acceptedAnswer":{"@type":"Answer","text":"Use Harbor's built-in langgraph agent, selected with --agent langgraph, and define a langgraph.json file that lists dependencies and maps a graph name to a make_graph factory function in agent.py. Harbor copies the agent into each trial's sandbox, installs the listed dependencies into a fresh virtual environment, and calls make_graph to build and run the compiled graph, keeping the integration to a single entry point. Teams wiring agents into eval pipelines can track integration guides like this one on daily.dev."}},{"@type":"Question","name":"How can I make my Harbor agent factory function model-agnostic instead of hardcoding a model?","acceptedAnswer":{"@type":"Answer","text":"Define make_graph to accept a config argument and read config[\"configurable\"][\"model\"] instead of hardcoding a model string. Harbor places whatever model is passed via the --model command-line flag into configurable.model, so the same factory function works across different models without code changes, as shown with create_deep_agent(model=config[\"configurable\"][\"model\"]). daily.dev helps developers keep up with practical patterns for building model-agnostic agent code."}},{"@type":"Question","name":"What does the harbor-langsmith plugin do when running agent evaluations?","acceptedAnswer":{"@type":"Answer","text":"Enabled with the --plugin langsmith flag, it records every Harbor job to LangSmith by syncing the dataset, creating an experiment, and logging a run per trial with the verifier's reward as feedback. If the agent under test supports LangSmith tracing, full step-by-step traces attach directly to the experiment; if not, the dataset, experiment, results, and feedback are still logged. Engineers evaluating agent behavior beyond pass or fail scores can follow tooling updates like this via daily.dev."}}]} ```