Master-Replica architecture is a foundational pattern in distributed systems. When a replica fails, it reconnects to the master and resumes replication from the last persisted sequence number. When the master fails, recovery involves three steps: detecting the crash (via heartbeat or failure detectors like Phi Accrual), setting up a new master (manually or via automated leader election among replicas), and announcing the new master to clients. A key challenge is data loss if the master crashes before propagating writes to replicas. The recommended solution is a synchronously replicated passive standby node that stays in sync with the master, enabling safe promotion without data loss or split-brain issues.
