Group of Bonds

Introduction

Group of Bonds, which is a logical group of existing Bond interfaces, provides additional link redundancy.

Example topology with Group of Bonds:

• There is one router - R
• There are two switches that connect to the router R - SW-1 and SW-2
• There are two Cluster Members GW-A (Active) and GW-B (Standby)
• There are two Bond interfaces on each Cluster Member - Bond-1 and Bond-2
• On the Cluster Member GW-A:
  • Bond-1 interface connects to the switch SW-1
  • Bond-2 interface connects to the switch SW-2
• On the Cluster Member GW-B:
  • Bond-1 interface connects to the switch SW-2
  • Bond-2 interface connects to the switch SW-1

Chain of events without Group of Bonds:

1. The Cluster Member GW-A is the Active and the Cluster Member GW-B is the Standby.
2. On the Cluster Member GW-A, the Bond-1 interface fails.
3. On the Cluster Member GW-A, the Critical Device Interface Active Check reports its state as "problem".
4. The Cluster Member GW-A changes its cluster state from Active to Down.
5. The cluster fails over - the Cluster Member GW-B changes its cluster state from Standby to Active.

This is not the desired behavior, because the Cluster Member GW-A connects not only to the switch SW-1, but also to the switch SW-2. In our example topology, there is no actual reason to fail-over from the Cluster Member GW-A to the Cluster Member GW-B.

In order to overcome this problem, Cluster Members use the Group of Bonds consisting of Bond-1 and Bond-2. The Group of Bonds fails only when both Bond interfaces fail on the Cluster Member. Only then the cluster fails over.

Chain of events with configured Group of Bonds:

1. The Cluster Member GW-A is the Active and the Cluster Member GW-B is the Standby.
2. On the Cluster Member GW-A, the Bond-1 interface fails.
3. On the Cluster Member GW-A, the Critical Device Interface Active Check reports its state as "problem".
4. The Cluster Member GW-A does not change its cluster state from Active to Down.
5. On the Cluster Member GW-A, the Bond-2 interface fails as well.
6. The Cluster Member GW-A changes its cluster state from Active to Down.
7. The cluster fails over - the Cluster Member GW-B changes its cluster state from Standby to Active.

To create a new Group of Bonds

Important - You must configure all Cluster Members in the same way.

1. Connect to the command line on the Cluster Member.
2. Log in to the Expert mode.
3. In VSX Cluster, switch to the context of the applicable Virtual System:

   [Expert@Member:0]# vsenv <VSID>

4. Modify the current $FWDIR/boot/modules/fwkern.conf file:
   1. Backup the current file:

      [Expert@Member:0]# cp -v $FWDIR/boot/modules/fwkern.conf{,_BKP}

   2. Edit the current file:

      [Expert@Member:0]# vi $FWDIR/boot/modules/fwkern.conf

   3. Add these two lines at the bottom of the file (spaces or comments are not allowed):
      • fwha_group_of_bonds_str=<Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>

      Example:

      fwha_group_of_bonds_str=GoB0:bond0,bond1;GoB1:bond2,bond3

      • fwha_arp_probe_method=1

      This parameter configures the Cluster Member to use the Virtual IP address as the Source IP address in the ARP Requests during the probing of the local network.

   4. Save the changes and exit the Vi editor.
5. Change the value of the kernel parameter fwha_group_of_bonds_str to add the desired Group of Bonds on-the-fly:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str '<Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>'

   Notes:

   • The apostrophe characters are mandatory part of the syntax.
   • Spaces are not allowed in the value of the kernel parameter fwha_group_of_bonds_str.

   Example:

   fw ctl set str fwha_group_of_bonds_str 'GoB0:bond0,bond1;GoB1:bond2,bond3'

6. Change the value of the kernel parameter fwha_arp_probe_method on-the-fly:

   [Expert@Member:0]# fw ctl set int fwha_arp_probe_method 1

7. Make sure the Cluster Member accepted the new configuration:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

   [Expert@Member:0]# fw ctl get int fwha_arp_probe_method

8. In SmartConsole, install the Access Control Policy on the cluster object.

To add a Bond interface to the existing Group of Bonds

Important - You must configure all Cluster Members in the same way.

1. Connect to the command line on the Cluster Member.
2. In VSX Cluster, switch to the context of the applicable Virtual System:

   [Expert@Member:0]# vsenv <VSID>

3. Log in to the Expert mode.
4. Modify the current $FWDIR/boot/modules/fwkern.conf file:
   1. Backup the current file:

      [Expert@Member:0]# cp -v $FWDIR/boot/modules/fwkern.conf{,_BKP}

   2. Edit the current file:

      [Expert@Member:0]# vi $FWDIR/boot/modules/fwkern.conf

   3. Edit the value of the kernel parameter fwha_group_of_bonds_str to add the desired Bond interface to the existing Group of Bonds.

      Example:

      fwha_group_of_bonds_str=GoB0:bond0,bond1;GoB1:bond2,bond3,bond4

   4. Save the changes and exit the Vi editor.
5. Get the current value of the kernel parameter fwha_group_of_bonds_str and copy it:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

6. Reset the current value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str ''

7. Make sure the Cluster Member reset the value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

8. Change the value of the kernel parameter fwha_group_of_bonds_str to add the desired Bond interface to the existing Group of Bonds on-the-fly:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str '<Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>'

   Notes:

   • The apostrophe characters are mandatory part of the syntax.
   • Spaces are not allowed in the value of the kernel parameter fwha_group_of_bonds_str.

   Example:

   fw ctl set str fwha_group_of_bonds_str 'GoB0:bond0,bond1;GoB1:bond2,bond3,bond4'

9. Make sure the Cluster Member accepted the new configuration:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

10. In SmartConsole, install the Access Control Policy on the cluster object.

To remove a Bond interface from the existing Group of Bonds

Important - You must configure all Cluster Members in the same way.

1. Connect to the command line on the Cluster Member.
2. Log in to the Expert mode.
3. In VSX Cluster, switch to the context of the applicable Virtual System:

   [Expert@Member:0]# vsenv <VSID>

4. Modify the current $FWDIR/boot/modules/fwkern.conf file:
   1. Backup the current file:

      [Expert@Member:0]# cp -v $FWDIR/boot/modules/fwkern.conf{,_BKP}

   2. Edit the current file:

      [Expert@Member:0]# vi $FWDIR/boot/modules/fwkern.conf

   3. Edit the value of the kernel parameter fwha_group_of_bonds_str to remove the desired Bond interface from the existing Group of Bonds.

      Example:

      fwha_group_of_bonds_str=GoB0:bond0,bond1;GoB1:bond2,bond3

   4. Save the changes and exit the Vi editor.
5. Get the current value of the kernel parameter fwha_group_of_bonds_str and copy it:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

6. Reset the current value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str ''

7. Make sure the Cluster Member reset the value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

8. Change the value of the kernel parameter fwha_group_of_bonds_str to remove the desired Bond interface from the existing Group of Bonds on-the-fly:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str '<Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>'

   Notes:

   • The apostrophe characters are mandatory part of the syntax.
   • Spaces are not allowed in the value of the kernel parameter fwha_group_of_bonds_str.

   Example:

   fw ctl set str fwha_group_of_bonds_str 'GoB0:bond0,bond1;GoB1:bond2,bond3

9. Make sure the Cluster Member accepted the new configuration:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

10. In SmartConsole, install the Access Control Policy on the cluster object.

To delete a Group of Bonds

Important - You must configure all Cluster Members in the same way.

1. Connect to the command line on the Cluster Member.
2. Log in to the Expert mode.
3. In VSX Cluster, switch to the context of the applicable Virtual System:

   [Expert@Member:0]# vsenv <VSID>

4. Modify the current $FWDIR/boot/modules/fwkern.conf file:
   1. Backup the current file:

      [Expert@Member:0]# cp -v $FWDIR/boot/modules/fwkern.conf{,_BKP}

   2. Edit the current file:

      [Expert@Member:0]# vi $FWDIR/boot/modules/fwkern.conf

   3. Delete these two lines in the file:
      • fwha_group_of_bonds_str=<Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>
      • fwha_arp_probe_method=1
   4. Save the changes and exit the Vi editor.
5. Reset the current value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl set str fwha_group_of_bonds_str ''

6. Make sure the Cluster Member reset the value of the kernel parameter fwha_group_of_bonds_str:

   [Expert@Member:0]# fw ctl get str fwha_group_of_bonds_str

7. In SmartConsole, install the Access Control Policy on the cluster object.

Monitoring

To see the configured Groups of Bonds, run the cphaprob show_bond_groups command.

Logs

Cluster Members generate some applicable logs, which you can see in:

• In non-VSX Cluster: In the /var/log/messages files and output of the dmesg command.
• In VSX Cluster: In the $FWDIR/log/fwk.elg file in the context of the applicable Virtual System.

To see the full logs, you need to enable the kernel debug:

• In the kernel module fw, debug flags error and ioctl
• Kernel module cluster, debug flag if

The full logs show:

• A physical slave interface goes down/up.
• A Bond interface goes down/up (regardless of a change in the Cluster Member's state).
• A Group of Bonds goes down/up.

Note - These logs are generated only once per event.

Limitations

• The maximal length on the text string <Name for Group of Bonds> is 16 characters.
• The maximal length on this text string is 1024 characters:

  <Name for Group of Bonds>:<List of all Bonds in this Group separated by comma>

• You can configure the maximum of five Groups of Bonds on a Cluster Member or Virtual System.
• You can configure the maximum of five Bond interfaces in each Groups of Bonds.
• Group of Bonds feature does support Virtual Switches and Virtual Routers. Meaning, do not configure Groups of Bonds in the context of these Virtual Devices.
• Group of Bonds feature supports only Bond interfaces that belong to the same Virtual System.

  You cannot configure bonds that belong to different Virtual Systems into the same Group of Bonds.

  You must perform all configuration in the context of the applicable Virtual System.

• Group of Bonds feature does support Sync interfaces (an interface on a Cluster Member, whose Network Type was set as Sync or Cluster+Sync in SmartConsole in cluster object).
• Group of Bonds feature does support Bridge interfaces.
• If a Bond interface goes down on one Cluster Member, the cphaprob show_bond_groups command on the peer Cluster Members also shows the same Bond interface as DOWN.

  This is because the peer Cluster Members stop receiving the CCP packets on that Bond interface and cannot probe the local network to determine that their Bond interface is really working.

• After you add a Bond interface to the existing Group of Bonds, you must install the Access Control Policy on the cluster object.
• After you remove a Bond interface from the existing Group of Bonds, you must install the Access Control Policy on the cluster object.