<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d" -->

---
title: Google found a way to test Gemini without seeing the...
description: Google DeepMind piloted what it calls the first double-blind evaluation of a proprietary frontier AI model, using confidential computing so evaluators never...
canonical: https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Google found a way to test Gemini without seeing the questions | daily.dev
og:description: Google DeepMind piloted what it calls the first double-blind evaluation of a proprietary frontier AI model, using confidential computing so evaluators never...
og:url: https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d
og:image: https://api.daily.dev/og/posts/okxpAEy4D.png
og:image:alt: Google found a way to test Gemini without seeing the questions
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Google found a way to test Gemini without seeing the questions

**[The New Stack](https://daily.dev/sources/newstack)** · 5 min read · 2 upvotes · 0 comments

## Summary

Google DeepMind piloted what it calls the first double-blind evaluation of a proprietary frontier AI model, using confidential computing so evaluators never see Gemini's weights and Google never sees the benchmark questions. The test ran Gemini 2.5 Flash Lite against private benchmarks from MLCommons and Singapore's AI Safety Institute inside a secure enclave built on Google Cloud Confidential Space, an NVIDIA H100 Confidential GPU, and Intel TDX host memory encryption. OpenMined's PySyft restricts network access to prevent code inside the enclave from leaking data, and remote attestation confirms both sides are running agreed-upon software before data is shared. Overhead is under 5%, though legal agreements and code reviews between organizations remain bigger hurdles than compute cost. Researchers are exploring multi-GPU clusters (H100, B200) to eventually evaluate larger models that don't fit on a single GPU.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://thenewstack.io/google-double-blind-evaluation>

## Questions this post answers

### What hardware does Google's double-blind AI evaluation pilot use to protect both model weights and test questions?

The pilot uses Google Cloud Confidential Space combined with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption. Model weights are stored in hardware-encrypted GPU memory while evaluation prompts sit in encrypted host memory, so neither Google nor the evaluator can access the other's protected data during the test.

_Anyone weighing trust and infrastructure trade-offs in AI evaluation can track hardware-based approaches like this on daily.dev._

### What model did Google test in its first double-blind AI evaluation and who provided the benchmarks?

Gemini 2.5 Flash Lite was tested against private benchmarks from MLCommons and the Singapore AI Safety Institute. The pilot ran on a single H100 80GB Confidential GPU, with researchers already exploring clusters of H100 and B200 GPUs to eventually evaluate larger models that don't fit on one GPU.

_Developers deciding how much to trust benchmark scores can follow evaluation methodology news on daily.dev._

### How much performance overhead does confidential computing add to AI model evaluation, according to Google's technical report?

The overhead is less than 5%, according to Google's technical report on the double-blind evaluation pilot. The bigger obstacles to running these evaluations are not compute costs but legal agreements and code reviews required between the organizations involved.

_Teams evaluating confidential computing trade-offs for AI workloads can follow developments like this on daily.dev._

## Similar posts on daily.dev

- [Gemini sovereign enterprise search on Google Distributed Cloud](https://daily.dev/posts/gemini-sovereign-enterprise-search-on-google-distributed-cloud-ty71g4s7p) · Google Cloud · 0 upvotes · 0 comments
- [Private AI Compute Enables Google Inference with Hardware Isolation and Ephemeral Data Design](https://daily.dev/posts/private-ai-compute-enables-google-inference-with-hardware-isolation-and-ephemeral-data-design-rhhxudjwb) · InfoQ · 1 upvotes · 0 comments

---

Tags: [#ai](https://daily.dev/tags/ai), [#data-science](https://daily.dev/tags/data-science), [#google-deepmind](https://daily.dev/tags/google-deepmind), [#google-gemini](https://daily.dev/tags/google-gemini)

[View this post on daily.dev](https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Google found a way to test Gemini without seeing the questions","url":"https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d"},"datePublished":"2026-08-27T20:53:51.937Z","dateModified":"2026-10-15T02:24:11.258Z","description":"Google DeepMind piloted what it calls the first double-blind evaluation of a proprietary frontier AI model, using confidential computing so evaluators never...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/46ea0fccf9cbb669ce706e74cbcd24e3?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/46ea0fccf9cbb669ce706e74cbcd24e3?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"The New Stack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"The New Stack","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/newstack","url":"https://daily.dev/sources/newstack"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,data-science,google-deepmind,google-gemini","timeRequired":"PT5M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"The New Stack","item":"https://daily.dev/sources/newstack"},{"@type":"ListItem","position":3,"name":"Google found a way to test Gemini without seeing the questions"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/google-found-a-way-to-test-gemini-without-seeing-the-questions-okxpaey4d#faq","mainEntity":[{"@type":"Question","name":"What hardware does Google's double-blind AI evaluation pilot use to protect both model weights and test questions?","acceptedAnswer":{"@type":"Answer","text":"The pilot uses Google Cloud Confidential Space combined with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption. Model weights are stored in hardware-encrypted GPU memory while evaluation prompts sit in encrypted host memory, so neither Google nor the evaluator can access the other's protected data during the test. Anyone weighing trust and infrastructure trade-offs in AI evaluation can track hardware-based approaches like this on daily.dev."}},{"@type":"Question","name":"What model did Google test in its first double-blind AI evaluation and who provided the benchmarks?","acceptedAnswer":{"@type":"Answer","text":"Gemini 2.5 Flash Lite was tested against private benchmarks from MLCommons and the Singapore AI Safety Institute. The pilot ran on a single H100 80GB Confidential GPU, with researchers already exploring clusters of H100 and B200 GPUs to eventually evaluate larger models that don't fit on one GPU. Developers deciding how much to trust benchmark scores can follow evaluation methodology news on daily.dev."}},{"@type":"Question","name":"How much performance overhead does confidential computing add to AI model evaluation, according to Google's technical report?","acceptedAnswer":{"@type":"Answer","text":"The overhead is less than 5%, according to Google's technical report on the double-blind evaluation pilot. The bigger obstacles to running these evaluations are not compute costs but legal agreements and code reviews required between the organizations involved. Teams evaluating confidential computing trade-offs for AI workloads can follow developments like this on daily.dev."}}]}
```

