Pacemaker : Add or Remove Nodes
2023/12/05 |
Add new nodes to an existing cluster.
As an example, add [node03] to the cluster like follows.
+--------------------+ | [ ISCSI Target ] | | dlp.srv.world | +----------+---------+ 10.0.0.30| | +----------------------+ | +----------------------+ | [ Cluster Node#1 ] |10.0.0.51 | 10.0.0.52| [ Cluster Node#2 ] | | node01.srv.world +----------+----------+ node02.srv.world | +----------------------+ | +----------------------+ | |10.0.0.53 +-----------------------+ | [ Cluster Node#3 ] | + node03.srv.world | +-----------------------+ |
[1] | |
[2] | Add a new node to an existing cluster. |
[root@node01 ~]# pcs status Cluster name: ha_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: node01.srv.world (version 2.1.6-10.1.el9-6fdc9deea29) - partition with quorum * Last updated: Tue Dec 5 14:37:00 2023 on node01.srv.world * Last change: Tue Dec 5 14:33:19 2023 by root via cibadmin on node01.srv.world * 2 nodes configured * 1 resource instance configured Node List: * Online: [ node01.srv.world node02.srv.world ] Full List of Resources: * scsi-shooter (stonith:fence_scsi): Started node01.srv.world Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled # authorize new node [root@node01 ~]# pcs host auth node03.srv.world
Username: hacluster
Password:
node03.srv.world: Authorized
# add new node [root@node01 ~]# pcs cluster node add node03.srv.world No addresses specified for host 'node03.srv.world', using 'node03.srv.world' Disabling sbd... node03.srv.world: sbd disabled Sending 'corosync authkey', 'pacemaker authkey' to 'node03.srv.world' node03.srv.world: successful distribution of the file 'corosync authkey' node03.srv.world: successful distribution of the file 'pacemaker authkey' Sending updated corosync.conf to nodes... node01.srv.world: Succeeded node02.srv.world: Succeeded node03.srv.world: Succeeded node01.srv.world: Corosync configuration reloaded |
[3] | Update setting of Fence Device. If SCSI fencing is configured for the fence device as in this example, log in to the shared storage for the fence device on the newly added node and install the SCSI fence agent ([2], [3] ). Then update the fencing device configuration as follows. |
# update fencing device list [root@node01 ~]# pcs stonith update scsi-shooter pcmk_host_list="node01.srv.world node02.srv.world node03.srv.world" [root@node01 ~]# pcs stonith config scsi-shooter Resource: scsi-shooter (class=stonith type=fence_scsi) Attributes: scsi-shooter-instance_attributes devices=/dev/disk/by-id/wwn-0x6001405f89df433c1ce4390afc6e0bad pcmk_host_list="node01.srv.world node02.srv.world node03.srv.world" Meta Attributes: scsi-shooter-meta_attributes provides=unfencing Operations: monitor: scsi-shooter-monitor-interval-60s interval=60s |
[4] | If you have already configured resources in your existing cluster, you need configure them for each resource so that the newly added node can successfully become active in the event of a failover. For example, if you have configured LVM shared storage as shown here, you will need to make the newly added node aware of its LVM shared storage beforehand. |
[root@node03 ~]#
iscsiadm -m discovery -t sendtargets -p 10.0.0.30 10.0.0.30:3260,1 iqn.2022-01.world.srv:dlp.target01 10.0.0.30:3260,1 iqn.2022-01.world.srv:dlp.target02
[root@node03 ~]#
iscsiadm -m node --login --target iqn.2022-01.world.srv:dlp.target02
[root@node03 ~]#
[root@node03 ~]# iscsiadm -m session -o show tcp: [1] 10.0.0.30:3260,1 iqn.2022-01.world.srv:dlp.target01 (non-flash) tcp: [2] 10.0.0.30:3260,1 iqn.2022-01.world.srv:dlp.target02 (non-flash) lvmdevices --adddev /dev/sdb1 [root@node03 ~]# lvm pvscan --cache --activate ay pvscan[17327] PV /dev/vda2 online, VG cs is complete. pvscan[17327] PV /dev/sdb1 ignore foreign VG. pvscan[17327] VG cs run autoactivation. 2 logical volume(s) in volume group "cs" now active |
[5] |
If you have already configured resources in your existing cluster, you need configure them for each resource so that the newly added node can successfully become active in the event of a failover. |
[6] | After completing all settings for each resource, start the cluster service on the newly added node. |
# start cluster services [root@node01 ~]# pcs cluster start node03.srv.world node03.srv.world: Starting Cluster... [root@node01 ~]# pcs cluster enable node03.srv.world node03.srv.world: Cluster Enabled pcs status Cluster name: ha_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: node02.srv.world (version 2.1.6-10.1.el9-6fdc9deea29) - partition with quorum * Last updated: Tue Dec 5 15:31:11 2023 on node01.srv.world * Last change: Tue Dec 5 15:30:11 2023 by hacluster via crmd on node02.srv.world * 3 nodes configured * 5 resource instances configured Node List: * Online: [ node01.srv.world node02.srv.world node03.srv.world ] Full List of Resources: * scsi-shooter (stonith:fence_scsi): Started node01.srv.world * Resource Group: ha_group: * lvm_ha (ocf:heartbeat:LVM-activate): Started node01.srv.world * httpd_fs (ocf:heartbeat:Filesystem): Started node01.srv.world * httpd_vip (ocf:heartbeat:IPaddr2): Started node01.srv.world * website (ocf:heartbeat:apache): Started node01.srv.world Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled |
[7] | Run fencing and verify that it successfully fails over to the newly added node. |
[root@node03 ~]#
[root@node03 ~]# pcs stonith fence node01.srv.world Node: node01.srv.world fenced pcs status Cluster name: ha_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: node03.srv.world (version 2.1.6-10.1.el9-6fdc9deea29) - partition with quorum * Last updated: Tue Dec 5 16:18:09 2023 on node01.srv.world * Last change: Tue Dec 5 16:15:44 2023 by hacluster via crmd on node03.srv.world * 3 nodes configured * 5 resource instances configured Node List: * Online: [ node01.srv.world node02.srv.world node03.srv.world ] Full List of Resources: * scsi-shooter (stonith:fence_scsi): Started node03.srv.world * Resource Group: ha_group: * lvm_ha (ocf:heartbeat:LVM-activate): Started node03.srv.world * httpd_fs (ocf:heartbeat:Filesystem): Started node03.srv.world * httpd_vip (ocf:heartbeat:IPaddr2): Started node03.srv.world * website (ocf:heartbeat:apache): Started node03.srv.world Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled |
[8] | To delete a node, run like follows. |
[root@node01 ~]# pcs status Cluster name: ha_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: node02.srv.world (version 2.1.6-10.1.el9-6fdc9deea29) - partition with quorum * Last updated: Tue Dec 5 15:31:11 2023 on node01.srv.world * Last change: Tue Dec 5 15:30:11 2023 by hacluster via crmd on node02.srv.world * 3 nodes configured * 5 resource instances configured Node List: * Online: [ node01.srv.world node02.srv.world node03.srv.world ] Full List of Resources: * scsi-shooter (stonith:fence_scsi): Started node01.srv.world * Resource Group: ha_group: * lvm_ha (ocf:heartbeat:LVM-activate): Started node01.srv.world * httpd_fs (ocf:heartbeat:Filesystem): Started node01.srv.world * httpd_vip (ocf:heartbeat:IPaddr2): Started node01.srv.world * website (ocf:heartbeat:apache): Started node01.srv.world Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled[root@node01 ~]# pcs cluster node remove node03.srv.world Destroying cluster on hosts: 'node03.srv.world'... node03.srv.world: Successfully destroyed cluster Sending updated corosync.conf to nodes... node01.srv.world: Succeeded node02.srv.world: Succeeded node01.srv.world: Corosync configuration reloaded # update fencing device list [root@node01 ~]# pcs stonith update scsi-shooter pcmk_host_list="node01.srv.world node02.srv.world"
pcs status Cluster name: ha_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: node02.srv.world (version 2.1.6-10.1.el9-6fdc9deea29) - partition with quorum * Last updated: Tue Dec 5 16:23:21 2023 on node01.srv.world * Last change: Tue Dec 5 16:22:57 2023 by root via cibadmin on node01.srv.world * 2 nodes configured * 5 resource instances configured Node List: * Online: [ node01.srv.world node02.srv.world ] Full List of Resources: * scsi-shooter (stonith:fence_scsi): Started node01.srv.world * Resource Group: ha_group: * lvm_ha (ocf:heartbeat:LVM-activate): Started node01.srv.world * httpd_fs (ocf:heartbeat:Filesystem): Started node01.srv.world * httpd_vip (ocf:heartbeat:IPaddr2): Started node01.srv.world * website (ocf:heartbeat:apache): Started node01.srv.world Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled |
Sponsored Link |