<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/build-a-jev-judge-1nss3ta1c" -->

---
title: Build a Jev Judge | daily.dev
description: A tutorial introduces Jev, TypeSafe AI's model for structured decision-making, as an alternative to generative LLM judges for evaluating AI agent behavior....
canonical: https://daily.dev/posts/build-a-jev-judge-1nss3ta1c
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Build a Jev Judge | daily.dev
og:description: A tutorial introduces Jev, TypeSafe AI's model for structured decision-making, as an alternative to generative LLM judges for evaluating AI agent behavior....
og:url: https://daily.dev/posts/build-a-jev-judge-1nss3ta1c
og:image: https://api.daily.dev/og/posts/1nSs3tA1c.png
og:image:alt: Build a Jev Judge
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Build a Jev Judge

**[Daily Dose of Data Science \| Avi Chawla \| Substack](https://daily.dev/sources/dailydoseofds)** · 13 min read · 0 upvotes · 0 comments

## Summary

A tutorial introduces Jev, TypeSafe AI's model for structured decision-making, as an alternative to generative LLM judges for evaluating AI agent behavior. Instead of generating text token by token, Jev takes a state (context/evidence) and predefined questions with typed answer formats (Noul, Score, Choice) and returns direct typed verdicts, reducing latency and cost for bounded, repeated judgments. The piece walks through building a refund-support agent evaluator that combines Jev for semantic judgments with Opik, an open-source experiment and observability platform, for recording and comparing results. It covers defining rubric questions, sending state to Jev's API, interpreting probability and confidence values correctly, wrapping Jev calls in a custom Opik metric, and running experiments with frozen synthetic traces. The piece stresses that Jev is best suited for focused, bounded judgments and should coexist with deterministic checks and generative LLM judges for cases needing deep reasoning or written explanations.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://blog.dailydoseofds.com/p/build-a-jev-judge>

## Questions this post answers

### What is Jev and how is it different from using an LLM as a judge for evaluating AI agent responses?

Jev is TypeSafe AI's model for making structured decisions: instead of generating text token by token like a typical LLM judge, it takes a state (context and evidence) plus a set of predefined questions and returns typed values directly from a bounded answer space, such as Noul, Score, or Choice. Because it skips token-by-token generation, it can offer lower latency and lower cost for focused, repeated evaluations, though an LLM judge remains better for cases needing detailed written explanations.

_daily.dev surfaces practical write-ups like this for teams comparing structured judges against LLM-as-judge setups._

### How do you interpret a Noul score of 0.98 from a structured AI judge like Jev?

A Noul value of 0.98 represents the probability assigned to the specific proposition being judged, not a percentage of how much of the answer is correct or grounded. A value near zero signals a strong 'no,' while a value near the middle indicates genuine uncertainty rather than a moderately good answer, so thresholds should be validated against labeled examples before production use.

_engineers building agent evaluation pipelines can track guidance like this on daily.dev before shipping thresholds._

### How can you combine Jev with Opik to build an agent evaluation pipeline for a support agent?

Send the agent's request, policy, tool call results, and final answer as a state to Jev along with typed rubric questions like grounded, action_honest, and helpfulness, then wrap the response in a custom Opik metric using score_result.ScoreResult to map Jev's answers into named scores. Opik then records, organizes, and lets you compare these judgments across experiments and agent versions.

_daily.dev helps developers stay on top of tooling patterns like pairing structured judges with observability platforms._

## Similar posts on daily.dev

- [What is Jev? How System One models could change AI agents : fireup.pro](https://daily.dev/posts/what-is-jev-how-system-one-models-could-change-ai-agents-fireup-pro-eqcqub8on) · FireUp · 1 upvotes · 0 comments

---

Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#llm](https://daily.dev/tags/llm)

[View this post on daily.dev](https://daily.dev/posts/build-a-jev-judge-1nss3ta1c)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Build a Jev Judge","url":"https://daily.dev/posts/build-a-jev-judge-1nss3ta1c","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/build-a-jev-judge-1nss3ta1c"},"datePublished":"2026-09-24T20:53:36.167Z","dateModified":"2026-10-15T02:22:31.776Z","description":"A tutorial introduces Jev, TypeSafe AI's model for structured decision-making, as an alternative to generative LLM judges for evaluating AI agent behavior....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/42e057917b19bedbb7c275c471b843b6?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/42e057917b19bedbb7c275c471b843b6?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Daily Dose of Data Science | Avi Chawla | Substack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Daily Dose of Data Science | Avi Chawla | Substack","logo":"https://media.daily.dev/image/upload/s--4IHQgTOw--/f_auto/v1710503712/logos/dailydoseofds","url":"https://daily.dev/sources/dailydoseofds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/build-a-jev-judge-1nss3ta1c","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,llm","timeRequired":"PT13M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Daily Dose of Data Science | Avi Chawla | Substack","item":"https://daily.dev/sources/dailydoseofds"},{"@type":"ListItem","position":3,"name":"Build a Jev Judge"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/build-a-jev-judge-1nss3ta1c#faq","mainEntity":[{"@type":"Question","name":"What is Jev and how is it different from using an LLM as a judge for evaluating AI agent responses?","acceptedAnswer":{"@type":"Answer","text":"Jev is TypeSafe AI's model for making structured decisions: instead of generating text token by token like a typical LLM judge, it takes a state (context and evidence) plus a set of predefined questions and returns typed values directly from a bounded answer space, such as Noul, Score, or Choice. Because it skips token-by-token generation, it can offer lower latency and lower cost for focused, repeated evaluations, though an LLM judge remains better for cases needing detailed written explanations. daily.dev surfaces practical write-ups like this for teams comparing structured judges against LLM-as-judge setups."}},{"@type":"Question","name":"How do you interpret a Noul score of 0.98 from a structured AI judge like Jev?","acceptedAnswer":{"@type":"Answer","text":"A Noul value of 0.98 represents the probability assigned to the specific proposition being judged, not a percentage of how much of the answer is correct or grounded. A value near zero signals a strong 'no,' while a value near the middle indicates genuine uncertainty rather than a moderately good answer, so thresholds should be validated against labeled examples before production use. engineers building agent evaluation pipelines can track guidance like this on daily.dev before shipping thresholds."}},{"@type":"Question","name":"How can you combine Jev with Opik to build an agent evaluation pipeline for a support agent?","acceptedAnswer":{"@type":"Answer","text":"Send the agent's request, policy, tool call results, and final answer as a state to Jev along with typed rubric questions like grounded, action_honest, and helpfulness, then wrap the response in a custom Opik metric using score_result.ScoreResult to map Jev's answers into named scores. Opik then records, organizes, and lets you compare these judgments across experiments and agent versions. daily.dev helps developers stay on top of tooling patterns like pairing structured judges with observability platforms."}}]}
```

