I Audited My AI Agent's Guardrails. Most of Them Weren't Running.
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
An audit of a personal AI coding agent setup found that five of seven supposed 'guardrails' weren't actually enforcing anything, despite looking correctly configured. Failure modes included set -e silently terminating guard scripts before their blocking logic ran, hook files that referenced plugin-scoped variables and went inert after uninstall, JSON field name mismatches causing hooks to silently no-op, and asymmetric directory discovery rules that kept archived agents loaded. The core lesson: instructions in CLAUDE.md/AGENTS.md are advisory context the model can choose to ignore, while hooks, CI checks, and wrapper scripts are deterministic code that fails loudly. The fix was building a 71-check test suite that actually triggers each guard with input it should reject, checks parity across multiple tools, and requires a new test whenever a new enforcement point is added.
Table of contents
Why is a rule in a config file not a guardrail?What does a silent failure actually look like?How do you find these before they cost you?What does a guardrail test suite look like?Does this mean instruction files are useless?FAQWhat to check in your own setupQuestions this post answers
Why does Claude Code sometimes ignore rules written in CLAUDE.md?
CLAUDE.md is delivered as context, not as a constraint the model is structurally unable to violate. The model weighs the instruction against everything else in the prompt and can judge a rule irrelevant to the current task. Long instruction files make this worse because content in the middle of a long context gets less attention (the lost-in-the-middle effect). Must-hold rules need a hook instead. Anyone hardening a Claude Code setup can compare hook and instruction-file patterns others have found on daily.dev.
Why should a bash guard script not use set -e?
Commands like grep, jq, diff, and test all return non-zero in normal, expected situations, such as grep finding no match. With set -e enabled, the script exits immediately at that line, before reaching its blocking logic, and the harness sees a script that exited cleanly rather than one that objected. The guard fails open precisely on the input it was written to catch. Use set -u or set -uo pipefail instead and handle errors explicitly. Developers debugging silent script failures often compare shell-safety patterns like this on daily.dev.
How can I tell if a hook or guardrail in my AI agent setup is actually firing?
Trigger it deliberately with input it should reject and confirm the action gets blocked; a clean run alone proves nothing, since a hook that never fires also produces a clean run. Common silent-failure causes include reading the wrong JSON field name across tool versions, hook config tied to a plugin variable that stops resolving after uninstall, and directory discovery rules that traverse differently across similar features. Teams auditing agent reliability track these kinds of enforcement gaps by following AI tooling discussions on daily.dev.