Trail of Bits shares three techniques their engineers developed for using Codex's /goal feature during the Patch the Planet bug-hunting initiative. Key findings include: letting Codex write its own goal prompts (using threat model files as input), defining outcomes rather than prescribing methods, and assigning one outcome per agent to avoid competing optimization targets. The post details a variant-analysis pipeline for the Rust compiler that found every Rust bug submitted, a CVE-to-Semgrep-rule conversion workflow that flagged 11 variant hits, and discovery of two potential privilege-escalation bugs in Keycloak's SAML component. The post emphasizes that human expert judgment remains essential for scoping, validating findings, and knowing what constitutes a reportable vulnerability.

8m read timeFrom blog.trailofbits.com
Post cover image
Table of contents
1. Let Codex write the goal2. Define the outcome, not the path3. Assign one outcome per agentWhere human judgment is needed
168 Impressions