Rolling Back a Failed Upgrade of a Security Group - Zero Downtime

This section describes the steps to roll back a failed upgrade of a Security Group from R81.20 with Zero Downtime.

This section describes the steps for rolling back a failed upgrade of a Security Group to R81.20.

This procedure supports only these downgrade paths for Security Groups:

  • from R81.20 to R81.10

  • from R81.20 to R81

Warnings:

  • Multi-Version Cluster (Zero Downtime) downgrade from R81.20 to R81.10 / R81 is not supported if a Security Group has Bond interfaces in the 802.3ad (LACP) mode on Uplink ports (Known Limitation PMTR-88191).

  • Before you follow the downgrade procedure, revert all changes in the topology you made after the upgrade procedure. For example, after the upgrade you added / removed interfaces, you changed the configuration of interfaces, you added / removed Security Group Members in the Security Group.

Important - While the Security Group still contains Security Group Members that run the R81.20 version, you can only run the script "sp_upgrade --revert" on the R81.20 Security Group Members.

Rolling Back If Only Some of the Security Group Members Were Upgraded

Important - Use this rollback procedure if you upgraded only some (not all) Security Group Members in the Security Group.

Step

Instructions

1

Connect to the command line on the Security Group.

2

If your default shell is /bin/bash (Expert mode), then go to the Gaia gClish:

gclish

3

Disable the SMO Image Cloning feature:

Note - The SMO Image Cloning feature automatically clones all the required software packages to the Security Group Members during their boot. When you install or remove software packages gradually on Security Group Members, it is necessary to disable this feature, so that after a reboot the updated Security Group Members do not clone the software packages from the existing non-updated Security Group Members.

  1. Examine the state of the SMO Image Cloning feature:

    show smo image auto-clone state

  2. Disable the SMO Image Cloning feature, if it is enabled:

    set smo image auto-clone state off

  3. Examine the state of the SMO Image Cloning feature:

    show smo image auto-clone state

4

Go to the Expert mode:

  • If your default shell is /bin/bash (Expert mode):

    exit

  • If your default shell is /etc/gclish (Gaia gClish):

    expert

5

Go to the context of one of the Security Group Members that were upgraded to R81.20:

member <Member ID>

Example:

member 1_1

6

Run the upgrade script with the "revert" parameter and follow the instructions on the screen:

sp_upgrade --revert

7

On each Security Group Member that was upgraded to R81.20, restore the Gaia automatic snapshot:

  1. Go to the context of each Security Group Member:

    member <Member ID>

    Example:

    member 1_2

  2. Go to Gaia Clish (do not use the Gaia gClish):

    clish

  3. Restore the Gaia automatic snapshot that was saved automatically before the upgrade.

    set snapshot revert AutoSnapShot_<Original-Version>_<Take>

    Example:

    set snapshot revert AutoSnapShot_AutoSnapShot_R81_47

  4. Wait for the Security Group Member to complete the reboot.

  5. Repeat Steps a-d for the next Security Group Member that was upgraded.

8

Connect to the command line on the Security Group.

9

If your default shell is /etc/gclish (Gaia gClish), then go to the Expert mode:

expert

10

Run the upgrade script with the "revert" parameter again and follow the instructions on the screen:

sp_upgrade --revert

11

Make sure the downgrade was successful:

asg diag verify

Rolling Back the Whole Security Group

Use this rollback procedure if you upgraded all Security Group Members in the Security Group and it is necessary to keep the current connections.

Important:

  • This procedure does not interrupt the traffic and does not require down time.

    However, this procedure takes more time comparing with the procedure Rolling Back a Failed Upgrade of a Security Group - Minimum Downtime.

  • In this rollback procedure, you divide all upgraded Security Group Members in a specific Security Group into two logical groups - denoted below as "A" and "B".

    You revert one logical group of the Security Group Members at one time.

    The other logical group of the Security Group Members continues to handle traffic.

    Each logical group should contain the same number of Security Group Members - as close as possible.

    Example 1:

    • There are 8 Security Group Members in the Security Group.

    • The Logical Group "A" contains Security Group Members from 1_1 to 1_4.

    • The Logical Group "B" contains Security Group Members from 1_5 to 1_8.

    Example 2:

    • There are 5 Security Group Members in the Security Group.

    • The Logical Group "A" contains Security Group Members from 1_1 to 1_3.

    • The Logical Group "B" contains Security Group Members 1_4 and 1_5.

Step

Instructions

1

Connect to the command line on the Security Group.

2

Go to the context of one of the Security Group Members in the Logical Group "A":

member <Member ID>

Example:

member 1_1

3

If your default shell is /etc/gclish (Gaia gClish), then go to the Expert mode:

expert

4

Run the upgrade script with the "revert" parameter and follow the instructions on the screen:

sp_upgrade --revert

5

Restore the Gaia automatic snapshot on each Security Group Member in the Logical Group "A" that was upgraded to R81.20:

  1. Go to the context of the Security Group Member:

    member <Member ID>

    Example:

    member 1_2

  2. Go to Gaia Clish (do not use the Gaia gClish):

    clish

  3. Restore the Gaia automatic snapshot that was saved automatically before the upgrade.

    set snapshot revert AutoSnapShot_<Original-Version>_<Take>

    Example:

    set snapshot revert AutoSnapShot_AutoSnapShot_R81_47

  4. Wait for the Security Group Member to complete the reboot.

  5. Repeat Steps a-d for the next Security Group Member in the Logical Group "A" that was upgraded.

6

Connect to the command line on the Security Group.

7

Go to the context of one of the Security Group Members in the Logical Group "A" that was downgraded from R81.20:

member <Member ID>

Example:

member 1_1

8

Run the upgrade script with the "revert" parameter again and follow the instructions on the screen:

sp_upgrade --revert

9

Restore the Gaia automatic snapshot on each Security Group Member in the Logical Group "B" that was upgraded to R81.20:

  1. Go to the context of the Security Group Member:

    member <Member ID>

    Example:

    member 1_5

  2. Go to Gaia Clish (do not use the Gaia gClish):

    clish

  3. Restore the Gaia automatic snapshot that was saved automatically before the upgrade.

    set snapshot revert AutoSnapShot_<Original-Version>_<Take>

    Example:

    set snapshot revert AutoSnapShot_AutoSnapShot_R81_47

  4. Wait for the Security Group Member to complete the reboot.

  5. Repeat Steps a-d for the next Security Group Member in the Logical Group "B" that was upgraded.

10

Connect to the command line on the Security Group.

11

If your default shell is /etc/gclish (Gaia gClish), then go to the Expert mode:

expert

12

Run the upgrade script with the "revert" parameter again and follow the instructions on the screen:

sp_upgrade --revert

13

Make sure the downgrade was successful:

asg diag verify

Rolling Back the Whole Security Group - With Downtime

Use this rollback procedure if you upgraded all Security Group Members in the Security Group and it is not necessary to keep the current connections.

Important - Schedule a maintenance window because this procedure interrupts all traffic that passes through the Security Group.

This rollback procedure save time because you revert all upgraded Security Group Members in a specific Security Group at the same time.

If traffic must not be interrupted, then follow the procedure Rolling Back a Failed Upgrade of a Security Group - Zero Downtime.

Step

Instructions

1

Connect to the command line on the Security Group.

2

If your default shell is /etc/gclish (Gaia gClish), then go to the Expert mode:

expert

3

Go from the Expert mode to Gaia gClish.

  • If your default shell is /etc/gclish (Gaia gClish), then run:

    exit

  • If your default shell is /etc/bash (Expert mode), then run:

    gclish

4

Restore the Gaia automatic snapshot that was saved automatically before the upgrade.

set snapshot revert AutoSnapShot_<Original-Version>_<Take>

Example:

set snapshot revert AutoSnapShot_AutoSnapShot_R81_47

5

Wait for the Security Group Members to complete the reboot.

6

Connect to the command line on the Security Group.

7

If your default shell is /etc/gclish (Gaia gClish), then go to the Expert mode:

expert

8

Run the upgrade script with the "revert" parameter and follow the instructions on the screen:

sp_upgrade --revert

9

Make sure the downgrade was successful:

asg diag verify