Tiberius is an open-source Java library for security and vulnerability testing of LLM applications, integrating with JUnit 5 and Spring Boot. It addresses the challenge of testing non-deterministic systems through five key capabilities: fixture-based regression testing (serializing scan results to JSON for reproducible downstream tests), guardrail validation against real attack data from fixtures, probabilistic security contracts using multi-trial scanning with statistical assertions, bias testing across demographic variants, and model fingerprinting to detect identity leakage. The library ships with 200+ probes across nine attack categories mapped to the OWASP LLM Top 10, supports custom mutation operators (Buffs), and integrates with Ollama, OpenAI, and Anthropic. It advocates for shared community attack datasets to raise the defensive baseline across industries.
