Upgrading a High Availability Deployment

Multi-Domain Security Management High Availability gives you management redundancy for all Domains. Multi-Domain Security Management High Availability operates at these levels:

• Multi-Domain Server High Availability - By default, Multi-Domain Servers are automatically synchronized with each other. One Multi-Domain Server is always defined as the Active Multi-Domain Server and all other Multi-Domain Servers are Standby Multi-Domain Servers. You can connect to an Active or Standby Multi-Domain Server to work on Domain management tasks.

  You can only do Global policy and global object management tasks using the active Multi-Domain Server. In the event that the active Multi-Domain Server is unavailable, you must change one of the standby Multi-Domain Servers to active.

• Domain Management Server High Availability - Multiple Domain Management Servers give Active/Standby redundancy for Domain management. One Domain Management Server for each Domain is Active. The other, fully synchronized Domain Management Servers for that Domain, are standbys. In the event that the Active Domain Management Server becomes unavailable, you must change one of the standby Domain Management Servers to active.

You can also use ClusterXL to give High Availability redundancy to your Domain Security Gateways. You use SmartConsole to configure and manage Security Gateway High Availability for Domain Management Servers.

Pre-Upgrade Verification and Tools

Run the pre-upgrade verification on all Multi-Domain Servers before upgrading any Multi-Domain Servers. Select the Pre-Upgrade Verification Only option from mds_setup. Upgrade the primary Multi-Domain Server only after you have fixed all errors and reviewed all warnings for all Multi-Domain Servers.

Multi-Domain Server High Availability

Multi-Domain Servers can only communicate and synchronize with other Multi-Domain Servers running the same version. If your deployment has more than one Multi-Domain Server, make sure they are upgraded to the same version.

To upgrade multiple Multi-Domain Servers:

1. Upgrade the primary Multi-Domain Server.
2. Upgrade the other Multi-Domain Servers.

During the upgrade process, we recommend that you do not use any of the Multi-Domain Servers to make changes to the databases. This can cause inconsistent synchronization between Multi-Domain Servers.

Important - Before you upgrade a Multi-Domain Server in High Availability Mode, all Domain Management Servers must be Active on the Primary Multi-Domain Server.

Note - You must upgrade your Multi-Domain Log Servers to the same version as the Multi-Domain Servers.

Upgrading Multi-Domain Servers and Domain Management Servers

To upgrade a Multi-Domain Server and a Domain Management Server:

1. Run pre-upgrade verification for all Multi-Domain Servers.
2. If a change to the global database is necessary, synchronize the Multi-Domain Servers immediately after making these changes. Update the database on one Multi-Domain Server and start synchronization. The other Multi-Domain Servers will get the database changes automatically.
3. If global database changes affect a global policy assigned to a Domain, assign the global policy again to all affected Domains.
4. If the verification command finds Domain Management Server level errors (for example, Gateways that are no longer supported by the new version):
   1. Make the required changes on the Active Domain Management Server.
   2. Synchronize the Active Domain Management Server with all Standby Domain Management Servers.
5. If a Domain has Log Servers:
   1. In the Domain SmartConsole, manually install the new database: select Policy > Install Database.
   2. Select all Log Servers.
   3. Make sure that the change to the Log Server is successful.

Note - When synchronizing, make sure that you have only one active Multi-Domain Server and one active Domain Management Server for each Domain.

Change the active Multi-Domain Server and Domain Management Server, and then synchronize the Standby computers.

Updating Objects in the Domain Management Server Databases

After upgrading the Multi-Domain Servers and Domain Management Servers, you must update the objects in all Domain Management Server databases. This is necessary because upgrade does not automatically update the object versions attribute in the databases. If you do not manually update the objects, the standby Domain Management Servers and Log Servers will show the outdated versions.

Update the objects with these steps on each Multi-Domain Server.

To update Domain Management Server and Log Server objects:

1. Make sure that all Domain Management Servers are up: mdsstat

   If a Domain Management Server is down, resolve the issue, and start the Domain Management Server: mdsstart_customer <DMSNAME>

2. Go to the top-level CLI: mdsenv
3. Run: $MDSDIR/scripts/mds_fix_cmas_clms_version -c ALL

   Optional: Update one Domain Management Server or Log Server at a time with this command:
   $MDSDIR/scripts/mds_fix_cmas_clms_version -c ALL -n <server_name>

4. After running the command and before synchronizing the Standby domains, run: mdsstop;mdsstart. See sk121718.
5. Synchronize all Standby Domain Management Servers.
6. Install the database in SmartConsole for the applicable Domain Management Server.

Managing Domain Management Servers During the Upgrade Process

The best practice is to avoid making any changes to Domain Management Server databases during the upgrade process. If your business model cannot support management down-time during the upgrade, you can continue to manage Domain Management Servers during the upgrade process.

This creates a risk of inconsistent Domain Management Server database content between instances on different Multi-Domain Servers. The synchronization process cannot resolve these inconsistencies.

After successfully upgrading one Multi-Domain Server, you can set its Domain Management Servers to Active while you upgrade the others. Synchronization between the Domain Management Servers occurs after all Multi-Domain Servers are upgraded.

If, during the upgrade process, you make changes to the Domain Management Server database using different Multi-Domain Servers, the contents of the two (or more) databases will be different. Because you cannot synchronize these databases, some of these changes will be lost. The Domain Management Server High Availability status appears as Collision.

You must decide which database version to retain and synchronize it to the other Domain Management Servers. You then must re-enter the lost changes to the synchronized database.