Azure Managed Redis (AMR) can significantly reduce LLM token costs and improve AI response times through semantic caching and agent memory. Unlike traditional exact key-value caching, semantic caching uses vector embeddings and cosine similarity search to match semantically equivalent prompts, serving cached responses without calling the LLM. A live demo shows cache hits on rephrased queries, configurable similarity thresholds, per-user vs. global cache namespacing to prevent data leakage, and a cost calculator showing ~70% savings at a 70% cache hit rate. A second demo covers agent memory: short-term conversation history and long-term user facts (preferences, interests, work) are stored in AMR with TTLs, enabling personalized multi-agent workflows where agents share durable context rapidly without repeated LLM calls.

โ€ข30m watch time
25 Impressions