In this guide, we will meticulously set up a PostgreSQL Patroni Cluster to achieve a highly available PostgreSQL environment. The installation process will be structured into several key steps:
- Etcd Cluster Installation
- PostgreSQL + Patroni Installation
- HAProxy + Keepalived Installation
- pgBackRest Backup Solution Setup
Our server infrastructure will consist of the following nodes:
- .190 → VIP
- .191 → HAProxy Primary Node
- .192 → HAProxy Secondary Node
- .193 → PostgreSQL Node 1
- .194 → PostgreSQL Node 2
- .195 → PostgreSQL Node 3
- .196 → Etcd Node 1
- .197 → Etcd Node 2
- .198 → Etcd Node 3
- .199 → Backup Node
We will commence with the first step: Etcd Cluster Installation. Etcd serves as a key-value store utilized by Patroni for distributed database management, and we will establish a three-node etcd cluster.
Step 1: Install Etcd Binaries
Execute the following commands on all etcd nodes:
dnf -y install curl wget vim
ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest | grep tag_name | cut -d '"' -f 4)
echo $ETCD_RELEASE
wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz
tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz
cd etcd-${ETCD_RELEASE}-linux-amd64
mv etcd* /usr/local/bin
ls /usr/local/bin
etcd --version
etcdctl version
etcdutl version
Step 2: Create Directories and Users
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
Step 3: Systemd Service File
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
Step 4: Etcd Configuration
For each etcd node, configure the etcd.conf file accordingly:
Etcd Node 1 (.196)
# cat /etc/etcd/etcd.conf [Member] ETCD_LISTEN_PEER_URLS="http://**.**.**.196:2380,http://localhost:2380" ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.196:2379" ETCD_DATA_DIR="/var/lib/etcd/default" [Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.196:2380" ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.196:2379" ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" ETCD_NAME="etcd1" ETCD_ELECTION_TIMEOUT=5000 ETCD_HEARTBEAT_INTERVAL=1000 ETCD_ENABLE_V2=true
Etcd Node 2 (.197)
# cat /etc/etcd/etcd.conf [Member] ETCD_LISTEN_PEER_URLS="http://**.**.**.197:2380,http://localhost:2380" ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.197:2379" ETCD_DATA_DIR="/var/lib/etcd/default" [Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.197:2380" ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.197:2379" ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" ETCD_NAME="etcd2" ETCD_ELECTION_TIMEOUT=5000 ETCD_HEARTBEAT_INTERVAL=1000 ETCD_ENABLE_V2=true
Etcd Node 3 (.198)
# cat /etc/etcd/etcd.conf [Member] ETCD_LISTEN_PEER_URLS="http://**.**.**.198:2380,http://localhost:2380" ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.198:2379" ETCD_DATA_DIR="/var/lib/etcd/default" [Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.198:2380" ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.198:2379" ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" ETCD_NAME="etcd3" ETCD_ELECTION_TIMEOUT=5000 ETCD_HEARTBEAT_INTERVAL=1000 ETCD_ENABLE_V2=true
Step 5: Start Etcd Service
systemctl enable etcd systemctl start etcd systemctl status etcd
Step 6: Verify Etcd Cluster Status
etcdctl member list etcdctl --endpoints=**.**.**.196:2379,**.**.**.197:2379,**.**.**.198:2379 endpoint status --write-out=table
With the Etcd cluster established, we will now proceed to the Patroni Installation Guide.
Step 1: Installing Required Packages
# apply each postgres node dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm dnf -qy module disable postgresql dnf install -y postgresql13-server dnf install -y postgresql13-contrib dnf install -y postgresql13-devel yum install patroni yum install python3-urllib3 yum install patroni-etcd.x86_64 cp -p /usr/share/doc/patroni/postgres0.yml /etc/patroni/patroni.yml
Step 2: Node 1 (.193) Configuration
# vi /etc/patroni/patroni.yml
----------------------------
scope: cluster
namespace: /var/lib/pgsql/config/
name: node1
restapi:
listen: **.**.**.193:8008
connect_address: **.**.**.193:8008
etcd:
host: **.**.**.196:2379
host: **.**.**.197:2379
host: **.**.**.198:2379
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
initdb:
- encoding: UTF8
- data-checksums
- waldir: /pg_wal
- wal-segsize=512
pg_hba:
- host replication replicator 127.0.0.1/32 scram-sha-256
- host replication replicator **.**.**.193/32 scram-sha-256
- host replication replicator **.**.**.194/32 scram-sha-256
- host replication replicator **.**.**.195/32 scram-sha-256
- host all all **.**.**.***/24 scram-sha-256
users:
admin:
password: admin
options:
- createrole
- createdb
postgresql:
listen: **.**.**.193:5432
connect_address: **.**.**.193:5432
data_dir: /pg_data/data
bin_dir: /usr/pgsql-13/bin
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: test123
superuser:
username: postgres
password: test123
create_replica_methods:
- pgbackrest
- basebackup
pgbackrest:
command: pgbackrest --stanza=pg_backup restore --type=none
keep_data: True
no_params: True
basebackup:
checkpoint: 'fast'
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false
Step 3: Node 2 (.194) Configuration
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
0
Step 4: Node 3 (.195) Configuration
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
0
Step 5: Start Patroni Service
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
2
Step 6: Check Cluster Status
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
3
Next, we will move on to the installation of HAProxy.
Step 1: Installing Required Package
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
4
Step 2: Configure HAProxy for Primary Node
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
5
Step 3: Configure HAProxy for Secondary Node
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
6
Step 4: Check HAProxy Status
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
7
Following the HAProxy setup, we will proceed with the installation of Keepalived.
Step 1: Installing Required Package
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
8
Step 2: Configure Keepalived for Primary Node
mkdir -p /var/lib/etcd/ mkdir /etc/etcd groupadd --system etcd useradd -s /sbin/nologin --system -g etcd etcd chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/
9
Step 3: Configure Keepalived for Secondary Node
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
0
Step 4: Check Keepalived Status
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
1
Finally, we will set up the pgBackRest backup solution.
Step 1: Installing the Required Package
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
2
Step 2: Configure pgBackRest on Backup Node
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
3
Step 3: Configure PostgreSQL Nodes
For each PostgreSQL node, ensure the pgBackRest settings are correctly configured in /etc/pgbackrest.conf.
Step 4: Initialize the pgBackRest Stanza on the Backup Node
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
4
Step 5: Run Full and Incremental Backups
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
5
Step 6: Check the Status of pgBackRest Backups
# vi /etc/systemd/system/etcd.service [Unit] Description=Etcd - Highly Available Key Value Store Documentation=man:etcd After=network.target Wants=network-online.target [Service] Environment=DAEMON_ARGS= Environment=ETCD_NAME=%H Environment=ETCD_DATA_DIR=/var/lib/etcd/default EnvironmentFile=/etc/etcd/etcd.conf Type=notify User=etcd PermissionsStartOnly=true ExecStart=/usr/local/bin/etcd $DAEMON_ARGS Restart=on-abnormal LimitNOFILE=65536 [Install] WantedBy=multi-user.target
6