Swiggy replaced a 150K-line deterministic decision tree for order cancellation support with a stateful multi-agent LLM system built on LangGraph. The new architecture separates intent classification (handled by the LLM) from operational truth (handled by deterministic backend APIs), uses Redis-backed checkpointing for multi-turn memory, and routes users to specialized flows based on detected intent. Key learnings include moving delay sub-classification out of the LLM to prevent hallucinations, introducing a repetition threshold to stop over-persuading users who clearly want to cancel, removing the LLM from the first turn to avoid caching issues, and humanizing retention responses by surfacing real operational costs. The system is monitored via automated daily reporting and real-time alerting for latency, throughput, and behavioral drift.
