Post

When failover isn’t safe: Building high-availability PostgreSQL on Kubernetes

Datadog describes how a simulated zonal failure with elevated network latency exposed a critical weakness in their PostgreSQL-on-Kubernetes architecture: asynchronous replication caused replicas to lag beyond safe promotion thresholds, making failover impossible without risking data loss. They rearchitected their clusters to use synchronous replication for failover candidates, coordinated by Patroni and ZooKeeper, ensuring that only replicas within an acceptable LSN distance can be promoted. The article explains how Patroni safely elects new leaders during network partitions, the trade-offs between availability and durability, and validation through benchmarking—practical guidance for operators managing stateful workloads on Kubernetes.

Read original article

This post is licensed under CC BY 4.0 by the author.