Advanced Cluster Configuration

A number of synchronization and ClusterXL capabilities are controlled by means of Security Gateway configuration parameters. Run these commands on the Security Gateway as follows:

fw ctl set int Parameter <value>

Parameter is any of the kernel parameters described in the following sections.

Changes to their default values must be implemented on all Cluster Members. Setting different values on Cluster Members can cause configuration problems and possibly connection failures.

Note - All these configuration parameters can be configured to survive a reboot.

How to Configure Reboot Survival

Security Gateway configuration parameters that are changed using the fw ctl set int command, do not survive reboot. To make them survive a reboot, you need to add the kernel parameter and its value to the $FWDIR/boot/modules/fwkern.conf file (see sk26202).

In the following instructions, Parameter is any of the parameters described in the following sections.

For general instructions, see Working with Kernel Parameters on Security Gateway.

Important - You must configure all the Cluster Members in the same way.

Controlling the Clustering and Synchronization Timers

The following Security Gateway configuration parameters are used to control the clustering and synchronization timers. Changing the default values is not recommended.

Clustering and Synchronization timers

Parameter	Meaning	Default Value
fwha_timer_cpha_res	The frequency of ClusterXL operations on the cluster.   Operations occur every: 10 multiplied by fwha_timer_cpha_res multiplied by fwha_timer_base_res milliseconds	1
fwha_timer_sync_res	The frequency of sync flush operations on the cluster.   Operations occur every: 10 multiplied by fwha_timer_sync_res multiplied by fwha_timer_base_res milliseconds	1
fwha_timer_base_res	Must be divisible by 10 with no remainders.	10

Blocking New Connections Under Load

This section applies only to Load Sharing modes. The reason for blocking new connections is that new connections are the main source of new Delta Synchronization traffic. Delta Synchronization may be at risk, if new traffic continues to be processed at high rate.

A related error message in cluster logs and in the /var/log/messages file is:

"FW-1: State synchronization is in risk. Please examine your synchronization network to avoid further problems!".

Reducing the amount of traffic passing through the Cluster Member protects the Delta Synchronization mechanism. See sk43896: Blocking New Connections Under Load in ClusterXL.

• The kernel parameter fw_sync_block_new_conns allows Cluster Member to detect heavy loads and start blocking new connections. Load is considered heavy when the synchronization transmit queue of the firewall starts to fill beyond the fw_sync_buffer_threshold.
  • To enable blocking new connections under load, set to 0.
  • To disable blocking new connections under load, set to -1 (0xFFFFFFFF hex) (default).

  Note that blocking new connections when sync is busy is only recommended for Load Sharing ClusterXL deployments. While it is possible to block new connections in High Availability mode, doing so does not solve inconsistencies in sync, as High Availability mode precludes that from happening. This parameter can be set to survive boot using the mechanism described in How to Configure a Kernel Parameter to Survive Reboot.

• The kernel parameter fw_sync_buffer_threshold configures the maximum percentage of the buffer that may be filled before new connections are blocked. By default it is set to 80, with a buffer size of 512. By default, if more than 410 consecutive packets are sent without getting an ACK on any one of them, new connections are dropped. When new connection blocking starts, fw_sync_block_new_conns is set to 0. When the situation is stable, it is set back to -1.
• The kernel parameter fw_sync_allowed_protocols is used to determine the type of connections that can be opened while the system is in a blocking state. Thus, the user can have better control over the system behavior in cases of unusual load. The fw_sync_allowed_protocols variable is a combination of flags, each specifying a different type of connection. The required value of the variable is the result of adding the separate values of these flags. For example, the default value of this variable is 24, which is the sum of TCP_DATA_CONN_ALLOWED (8) and UDP_DATA_CONN_ALLOWED (16), meaning that the default allows only TCP and UDP data connections to be opened under load.

Summary table:

ICMP_CONN_ALLOWED	1
TCP_CONN_ALLOWED	2 (except for data connections)
UDP_CONN_ALLOWED	4 (except for data connections)
TCP_DATA_CONN_ALLOWED	8 (the control connection should be established or allowed)
UDP_DATA_CONN_ALLOWED	16 (the control connection should be established or allowed)

Reducing the Number of Pending Packets

ClusterXL prevents out-of-state packets in non-sticky connections. It does this by holding packets until a Sync ACK is received from all other active Cluster Members. If for some reason a Sync ACK is not received, the Security Gateway on the Cluster Member will not release the packet, and the connection will not be established.

To find out if held packets are not being released, run the fw ctl pstat command. If the output of the command shows that the Number of Pending Packets is large under normal loads (more than 100 pending packets), and this value does not decrease over time, use the fwldbcast_pending_timeout parameter to reduce the number of pending packets.

Change the value of fwldbcast_pending_timeout from the default value of 50 to a value lower than 50.

The value is in ticks units, where each tick is equal to 0.1 sec, so that 50 ticks are 5 seconds.

The value represents the time, after which packets are released even if Sync ACKs are not received.