PostgreSQL with Patroni: Installation and Configuration

In this guide, we will meticulously set up a PostgreSQL Patroni Cluster to achieve a highly available PostgreSQL environment. The installation process will be structured into several key steps:

  1. Etcd Cluster Installation
  2. PostgreSQL + Patroni Installation
  3. HAProxy + Keepalived Installation
  4. pgBackRest Backup Solution Setup

Our server infrastructure will consist of the following nodes:

  • .190 → VIP
  • .191 → HAProxy Primary Node
  • .192 → HAProxy Secondary Node
  • .193 → PostgreSQL Node 1
  • .194 → PostgreSQL Node 2
  • .195 → PostgreSQL Node 3
  • .196 → Etcd Node 1
  • .197 → Etcd Node 2
  • .198 → Etcd Node 3
  • .199 → Backup Node

We will commence with the first step: Etcd Cluster Installation. Etcd serves as a key-value store utilized by Patroni for distributed database management, and we will establish a three-node etcd cluster.

Step 1: Install Etcd Binaries

Execute the following commands on all etcd nodes:

dnf -y install curl wget vim  
ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest | grep tag_name | cut -d '"' -f 4)  
echo $ETCD_RELEASE  
wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz  
tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz  
cd etcd-${ETCD_RELEASE}-linux-amd64  
mv etcd* /usr/local/bin  
ls /usr/local/bin  
etcd --version  
etcdctl version  
etcdutl version  

Step 2: Create Directories and Users

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

Step 3: Systemd Service File

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

Step 4: Etcd Configuration

For each etcd node, configure the etcd.conf file accordingly:

Etcd Node 1 (.196)

# cat /etc/etcd/etcd.conf  
[Member]  
ETCD_LISTEN_PEER_URLS="http://**.**.**.196:2380,http://localhost:2380"  
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.196:2379"  
ETCD_DATA_DIR="/var/lib/etcd/default"  
[Clustering]  
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.196:2380"  
ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.196:2379"  
ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380"  
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"  
ETCD_INITIAL_CLUSTER_STATE="new"  
ETCD_NAME="etcd1"  
ETCD_ELECTION_TIMEOUT=5000  
ETCD_HEARTBEAT_INTERVAL=1000  
ETCD_ENABLE_V2=true  

Etcd Node 2 (.197)

# cat /etc/etcd/etcd.conf  
[Member]  
ETCD_LISTEN_PEER_URLS="http://**.**.**.197:2380,http://localhost:2380"  
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.197:2379"  
ETCD_DATA_DIR="/var/lib/etcd/default"  
[Clustering]  
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.197:2380"  
ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.197:2379"  
ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380"  
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"  
ETCD_INITIAL_CLUSTER_STATE="new"  
ETCD_NAME="etcd2"  
ETCD_ELECTION_TIMEOUT=5000  
ETCD_HEARTBEAT_INTERVAL=1000  
ETCD_ENABLE_V2=true  

Etcd Node 3 (.198)

# cat /etc/etcd/etcd.conf  
[Member]  
ETCD_LISTEN_PEER_URLS="http://**.**.**.198:2380,http://localhost:2380"  
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://**.**.**.198:2379"  
ETCD_DATA_DIR="/var/lib/etcd/default"  
[Clustering]  
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://**.**.**.198:2380"  
ETCD_ADVERTISE_CLIENT_URLS="http://**.**.**.198:2379"  
ETCD_INITIAL_CLUSTER="etcd1=http://**.**.**.196:2380,etcd2=http://**.**.**.197:2380,etcd3=http://**.**.**.198:2380"  
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"  
ETCD_INITIAL_CLUSTER_STATE="new"  
ETCD_NAME="etcd3"  
ETCD_ELECTION_TIMEOUT=5000  
ETCD_HEARTBEAT_INTERVAL=1000  
ETCD_ENABLE_V2=true  

Step 5: Start Etcd Service

systemctl enable etcd  
systemctl start etcd  
systemctl status etcd  

Step 6: Verify Etcd Cluster Status

etcdctl member list  
etcdctl --endpoints=**.**.**.196:2379,**.**.**.197:2379,**.**.**.198:2379 endpoint status --write-out=table  

With the Etcd cluster established, we will now proceed to the Patroni Installation Guide.

Step 1: Installing Required Packages

# apply each postgres node  
dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm  
dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm  
dnf -qy module disable postgresql  
dnf install -y postgresql13-server  
dnf install -y postgresql13-contrib  
dnf install -y postgresql13-devel  
yum install patroni  
yum install python3-urllib3  
yum install patroni-etcd.x86_64  
cp -p /usr/share/doc/patroni/postgres0.yml /etc/patroni/patroni.yml  

Step 2: Node 1 (.193) Configuration

# vi /etc/patroni/patroni.yml  
----------------------------  
scope: cluster  
namespace: /var/lib/pgsql/config/  
name: node1  
restapi:  
    listen: **.**.**.193:8008  
    connect_address: **.**.**.193:8008  
etcd:  
    host: **.**.**.196:2379  
    host: **.**.**.197:2379  
    host: **.**.**.198:2379  
bootstrap:  
  dcs:  
    ttl: 30  
    loop_wait: 10  
    retry_timeout: 10  
    maximum_lag_on_failover: 1048576  
    postgresql:  
      use_pg_rewind: true  
      use_slots: true  
      parameters:  
  initdb:  
  - encoding: UTF8  
  - data-checksums  
  - waldir: /pg_wal  
  - wal-segsize=512  
  pg_hba:  
  - host replication replicator 127.0.0.1/32 scram-sha-256  
  - host replication replicator **.**.**.193/32 scram-sha-256  
  - host replication replicator **.**.**.194/32 scram-sha-256  
  - host replication replicator **.**.**.195/32 scram-sha-256  
  - host all all **.**.**.***/24 scram-sha-256  
  users:  
    admin:  
      password: admin  
      options:  
        - createrole  
        - createdb  
postgresql:  
  listen: **.**.**.193:5432  
  connect_address: **.**.**.193:5432  
  data_dir: /pg_data/data  
  bin_dir: /usr/pgsql-13/bin  
  pgpass: /tmp/pgpass  
  authentication:  
    replication:  
      username: replicator  
      password: test123  
    superuser:  
      username: postgres  
      password: test123  
  create_replica_methods:  
    - pgbackrest  
    - basebackup  
  pgbackrest:  
   command: pgbackrest --stanza=pg_backup restore --type=none  
   keep_data: True  
   no_params: True  
  basebackup:  
    checkpoint: 'fast'  
tags:  
    nofailover: false  
    noloadbalance: false  
    clonefrom: false  
    nosync: false  

Step 3: Node 2 (.194) Configuration

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

0

Step 4: Node 3 (.195) Configuration

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

0

Step 5: Start Patroni Service

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

2

Step 6: Check Cluster Status

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

3

Next, we will move on to the installation of HAProxy.

Step 1: Installing Required Package

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

4

Step 2: Configure HAProxy for Primary Node

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

5

Step 3: Configure HAProxy for Secondary Node

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

6

Step 4: Check HAProxy Status

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

7

Following the HAProxy setup, we will proceed with the installation of Keepalived.

Step 1: Installing Required Package

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

8

Step 2: Configure Keepalived for Primary Node

mkdir -p /var/lib/etcd/  
mkdir /etc/etcd  
groupadd --system etcd  
useradd -s /sbin/nologin --system -g etcd etcd  
chown -R etcd:etcd /var/lib/etcd/ && chmod 0775 /var/lib/etcd/  

9

Step 3: Configure Keepalived for Secondary Node

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

0

Step 4: Check Keepalived Status

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

1

Finally, we will set up the pgBackRest backup solution.

Step 1: Installing the Required Package

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

2

Step 2: Configure pgBackRest on Backup Node

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

3

Step 3: Configure PostgreSQL Nodes

For each PostgreSQL node, ensure the pgBackRest settings are correctly configured in /etc/pgbackrest.conf.

Step 4: Initialize the pgBackRest Stanza on the Backup Node

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

4

Step 5: Run Full and Incremental Backups

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

5

Step 6: Check the Status of pgBackRest Backups

# vi /etc/systemd/system/etcd.service  
[Unit]  
Description=Etcd - Highly Available Key Value Store  
Documentation=man:etcd  
After=network.target  
Wants=network-online.target  
[Service]  
Environment=DAEMON_ARGS=  
Environment=ETCD_NAME=%H  
Environment=ETCD_DATA_DIR=/var/lib/etcd/default  
EnvironmentFile=/etc/etcd/etcd.conf  
Type=notify  
User=etcd  
PermissionsStartOnly=true  
ExecStart=/usr/local/bin/etcd $DAEMON_ARGS  
Restart=on-abnormal  
LimitNOFILE=65536  
[Install]  
WantedBy=multi-user.target  

6

Tech Optimizer
PostgreSQL with Patroni: Installation and Configuration