Cachy: How we made our notebooks 60x faster. – Answer.AI
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Answer.AI open-sourced cachy, a Python package that caches LLM API responses by patching the httpx library's send method. With a single line of code, it automatically saves and replays HTTP responses, reducing test suite runtime from ~2 minutes to ~2 seconds (60x speedup). Benefits include faster CI/CD pipelines without needing API keys, deterministic test results, and cleaner notebook diffs. It supports async, streaming, and works with any httpx-based API beyond just LLM providers like Anthropic and OpenAI.
32 Impressions