This is a draft document that was built and uploaded automatically. It may document beta software and be incomplete or even incorrect. Use this document at your own risk.

Bootstrapping the MariaDB Galera Cluster with Pacemaker when a node is missing

Pacemaker does not promote a node to master until it received from all nodes the latest Sequence number (seqno). That is a problem when one node of the MariaDB Galera Cluster is down (eg. due to hardware or network problems) because the Sequence number can not be received from the unavailable node. To recover a MariaDB Galera Cluster manual steps are needed to select a bootstrap node for MariaDB Galera Cluster and to promote that node with Pacemaker.

Important

Selecting the correct bootstrap node (depending on the highest Sequence number (seqno)) is important. If the wrong node is selected data loss is possible.

  1. To find out which node has the latest Sequence number, call the following command on all MariaDB Galera Cluster nodes and select the node with the highest Sequence number.

    mysqld_safe --wsrep-recover
              tail -5 /var/log/mysql/mysqld.log
              ...
              [Note] WSREP: Recovered position: 7a477edc-757d-11e9-a01a-d218e7381711:2490

    At the end of /var/log/mysql/mysqld.log the Sequence number is written (in this example, the sequence number is 2490). After all Sequence numbers are collected from all nodes, the node with the highest Sequence number is selected for bootstrap node. In this example, the node with the highest Sequence number is called node1.

  2. Temporarily mark the galera Pacemaker resource as unmanaged:

              crm resource unmanage galera
            
  3. Mark the node as bootstrap node (call the following commands from the bootstrap node which is node1 in this example):

              crm_attribute -N node1 -l reboot --name galera-bootstrap -v true
              crm_attribute -N node1 -l reboot --name master-galera -v 100
            
  4. Promote the bootstrap node:

              crm_resource --force-promote -r galera -V
            
  5. Redetect the current state of the galera resource:

              crm resource cleanup galera
            
  6. Return the control to Pacemaker:

              crm resource manage galera
              crm resource start galera
            

The MariaDB Galera Cluster is now running and Pacemaker is handling the cluster.