Print Download PDF Send Feedback

Previous

Next

Optimizing VSX

In This Section:

QoS Enforcement

Monitoring Memory Resources

VSX CPU Monitoring Commands

SNMP Monitoring

Configuring Jumbo Frames

QoS Enforcement

QoS Enforcement for VSX provides the ability to control the network quality of service in the VSX network environment. QoS is based on the Differentiated Services architecture and allows assigning different transmission characteristics to different classes of service.

Differentiated Services is a computer networking architecture that specifies a simple, scalable and coarse-grained mechanism for classifying, managing network traffic and providing quality of service (QoS) guarantees on modern IP networks. Differential services can, for example, be used to provide low-latency, guaranteed service (GS) to critical network traffic such as voice or video while providing simple best-effort traffic guarantees to non-critical services such as web traffic or file transfers.

The major characteristics that are controllable by QoS are latency and bandwidth allocation. QoS is designed to provide QoS functionality with minimal impact on performance. QoS works seamlessly with Check Point Performance Pack.

The VSX network usually includes various types of traffic such as:

Without QoS Enforcement, all these different traffic types are given equal priority on the VSX Gateway and are handled in a simple FIFO (first in-first out) manner. When the VSX Gateway is congested, all traffic types suffer the same degree of latency and drops. Also, high-volume traffic may starve other types of low-volume traffic.

With QoS, the special requirements of each traffic type can be met. For example:

Note - QoS requires the use of DiffServ-enabled routers to mark preferred traffic types with a special tag. The tag is the DSCP (DiffServ Code Point), which represents the six most significant bits of the IP header's TOS field, as described in RFC 2474. The VSX Gateway should then be configured to give traffic with this tag the required priority.

Overview

QoS Enforcement for VSX provides the ability to control the network quality of service in the VSX network environment. QoS is based on the Differentiated Services architecture and allows assigning different transmission characteristics to different classes of service.

Differentiated Services is a computer networking architecture that specifies a simple, scalable and coarse-grained mechanism for classifying, managing network traffic and providing quality of service (QoS) guarantees on modern IP networks. Differential services can, for example, be used to provide low-latency, guaranteed service (GS) to critical network traffic such as voice or video while providing simple best-effort traffic guarantees to non-critical services such as web traffic or file transfers.

The major characteristics that are controllable by QoS are latency and bandwidth allocation. QoS is designed to provide QoS functionality with minimal impact on performance. QoS works seamlessly with Check Point Performance Pack.

The VSX network usually includes various types of traffic such as:

Without QoS Enforcement, all these different traffic types are given equal priority on the VSX Gateway and are handled in a simple FIFO (first in-first out) manner. When the VSX Gateway is congested, all traffic types suffer the same degree of latency and drops. Also, high-volume traffic may starve other types of low-volume traffic.

With QoS, the special requirements of each traffic type can be met. For example:

Note - QoS requires the use of DiffServ-enabled routers to mark preferred traffic types with a special tag. The tag is the DSCP (DiffServ Code Point), which represents the six most significant bits of the IP header's TOS field, as described in RFC 2474. The VSX Gateway should then be configured to give traffic with this tag the required priority.

Architecture

Three major aspects of the QoS architecture are:

Differentiated Services Support

QoS provides basic support for Differentiated Services, an architecture for specifying and controlling network traffic by class so that certain types of traffic receive priority over others. The differentiated services architecture PHB's (per-hop behaviors).

When marked packets arrive to the VSX machine, they are classified and prioritized according to their DSCP (differential services code-point) values. To enhance performance, QoS does not mark packets with DSCP and does not change their Type of Service (ToS) values. QoS instead relies on peripheral devices (namely routers) to mark packets with the appropriate ToS value.

Inbound Prioritization

While Differentiated Services support in routers is usually performed on outbound traffic, QoS for VSX prioritizes traffic on the inbound side because, in VSX deployments, QoS is primarily governed by system resources, namely the CPU, and not by network bandwidth.

To prevent the VSX machine from becoming a bottleneck in the network, prioritization is enforced when packets arrive at the VSX machine, and before CPU processing is assigned.

Inbound prioritization allows an earlier control on the loss and delay rate.

Policy with Global Scope

To minimize the impact of QoS functionality on performance, QoS is not performed on a per interface basis, but for the entire system. This means that a certain class of service will apply to all traffic entering the VSX Gateway or cluster, regardless of the specific interface from which the traffic originates.

Note - On multiple-CPU machines, enforcement is not performed system-wide, but executed per-CPU. This means that global enforcement is done separately on traffic processed by each CPU.

QoS Features

Two main features of QoS are:

Resource Allocation

System resources are allocated by assigning different weights to different classes of service. A weight is the relative portion of the available resources allocated to a class. Allocating resources according to weights ensures full utilization of the line even if a specific class is not using all of its resources. In such a case, the remaining resources are divided among the remaining classes in accordance with their relative weights.

Latency

For some types of traffic, such as voice and video, it is necessary to minimize the latency (delay) of packets. Latency is controlled by defining special LLQ (low-latency queuing) classes. These classes are handled in a strict priority manner. LLQ packets are handled immediately upon arrival, and before packets that do not belong to LLQ classes.

QoS supports multiple LLQ classes. In some cases, it may be necessary to define more than one Low Latency class, for example when different types of traffic have a different sensitivity to delays. In such cases, a class with the higher sensitivity to delay receives a higher priority than a class with the lower sensitivity.

Note - When LLQ classes are used, it is assumed that the expected traffic will not exceed a relatively small amount of the available resources. Although QoS does not allow LLQ traffic to starve non-LLQ traffic, too much LLQ traffic reduces overall network quality of service and prevents efficient management of weighted resources.

WRED

RED (Random Early Drop) is a congestion avoidance mechanism for detecting and preventing congestions. It takes advantage of TCP's congestion control mechanism by randomly dropping packets during periods of congestion. This causes TCP senders to slow down their transmission, thus preventing high congestion.

QoS implements WRED (Weighted RED) in which packets are dropped according to their priority. WRED mostly affects traffic which is of low priority and which exceeds its weight.

QoS Management

To manage the network quality of service it is necessary to create and install a QoS policy. The QoS policy consists of a list of up to 15 classes of service. Each class is assigned certain traffic characteristics and DSCP values.

The QoS policy is managed using the cpqos command.

Class of Service Definitions

The definition of a class of service includes the following:

Priority and LLQs

If there are multiple LLQ classes, packets are handled in a strict priority-based manner. Packets from a class with a higher priority are handled before packets with a lower priority class.

Priority and Drop Precedence

Priority also determines the probability of drops. A class with a lower priority has a higher drop precedence during times of congestion.

The class priority is not the only factor that determines if drops occur. Other factors affect drops, for example if the class is LLQ or if the class exceeds its assigned resource allocation.

LLQ's are not immune to drops. Although LLQ's are processed as soon as they arrive (and thus have a lower drop rate), drops may occur if there are many LLQ classes or if a large portion of the incoming traffic is LLQ.

QoS Configuration

All user interactions with the QoS module are performed with the cpqos command.

The cpqos Command

cpqos - Manage the network quality of service.
 
cpqos install            - Install the QoS policy.
cpqos uninstall          - Uninstall the QoS policy.
cpqos status             - Check if policy is installed.
cpqos class show [-b]    - Show the QoS policy. [-b] display dscp in binary numbers.
cpqos class add <name> prio <val> type <llq|reg> [weight <val>] dscp <val[,val2[,val3...]]>
                         - Add new class with specified name
                           prio   - priority (1-15)
                           type   - low-latency or regular
                           DSCP   - values "default" or 0-63 (decimal or 8 bits binary)
                           weight - (1-1000) for classes of type "reg"
cpqos class del <name>   - Delete specified class name.
cpqos stats [-u]         - Show statistics. [-u] will show statistics per CPU.

For cpqos:

cpqos class add

This command adds a class with the following arguments:

Argument

Value

name

Unique name for the class

priority

Value between 1 and 15. A low value indicates a higher priority

type

"llq" for low-latency classes or "reg" for regular, weighted classes

weight

This value is used only for classes of type "reg". It determines the relative portion of the resources that the class will receive in relation to other weighted classes. Valid values are between 0 and 1000.

dscp

The DiffServ code-points assigned to the class. Multiple DSCP's can be specified, separated by commas, with no spaces between values.

Values are in decimal (not binary format) with values from 0 to 63 or "default". There can be only one class with a "default" DSCP.

The default class is used for traffic without DiffServ marking (e.g. tos=0) or traffic with DSCP values that are not assigned to any other class. If no class is used as "default", all 64 DSCP values must be assigned to the classes. A DSCP value cannot be assigned to more than one class.

Note - Changes to the policy with cpqos class add are enforced only after the policy is installed.

cpqos class del

This command deletes the class of the specified name. Changes to the policy with cpqos class del are enforced only after the policy is installed.

cpqos class show

This command shows the classes defined in the QoS policy.

cpqos install

This command installs the previously created QoS policy. It also validates the overall integrity of the policy. Once installed, the policy remains installed even if the machine reboots.

cpqos uninstall

This command un-installs the previously installed QoS policy. If un-installed, the policy will not be installed again when the machine boots.

cpqos stats

This command shows QoS statistics. cpqos stats prints a line of statistics for each of the defined classes. Each line includes the following data columns:

Column

Value

rx

Number of bytes that arrived to this class since the last time statistics were presented

tx

Number of bytes that were transmitted from this class since the last time statistics were presented

drops

Number of bytes that were dropped from this class since the last time statistics were presented

Note:

QoS Policy File

The QoS policy file is qos_policy.C, located in the $FWDIR/database directory. The QoS policy file is created when the cpqos command is run for the first time. The QoS policy file should not be edited manually. Use cpqos class add/del to create entries. To maintain multiple QoS policies, rename qos_policy.C or copy it to another directory, and copy it back to $FWDIR/database/qos_policy.C when the policy needs to be enforced.

QoS Default Configuration

Default QoS configuration is set to "uninstall" (e.g. not enforced). Calling cpqos install or cpqos uninstall sets the default configuration after boot

Sample Differentiated Services Implementation

This section presents a sample differentiated services implementation. It includes examples for configuration, monitoring and statistics.

Sample Traffic Types

Traffic Type

Meaning...

Diamond

Real-time traffic (e.g. VOIP) which requires little bandwidth and is sensitive to latency and drops. This traffic is usually assigned to the EF (Expedited-Forwarding) PHB (Per Hop Behavior).

Platinum

Real-time traffic with low bandwidth requirements that is less sensitive to latency and drops than Diamond.

Gold

Traffic which is sensitive to drops

Silver

Traffic which is less sensitive to drops than Gold.

Bronze

Various types of traffic which require resource allocation. This traffic is usually assigned to the Best-Effort PHB.

Copper

High-volume traffic with a tendency to consume bandwidth

Configuration Guidelines

Your QoS policy should apply these guidelines:

Configuration Examples

This examples of the cpqos class add command creates classes for traffic of various types:

cpqos class add Diamond type llq prio 1 dscp 46
cpqos class add Platinum type llq prio 2 dscp 32
cpqos class add Gold type reg prio 3 weight 100 dscp 26
cpqos class add Silver type reg prio 4 weight 100 dscp 28
cpqos class add Bronze type reg prio 5 weight 200 dscp default
cpqos class add Copper type reg prio 15 weight 50 dscp 10,12,14

Monitoring example shows previously defined classes:

[Expert@cpmodule:0]# cpqos class show
class: Diamond
priority: 1
type: llq
weight: 0
DSCPs: 46

class: Platinum
priority: 2
type: llq
weight: 0
DSCPs: 32

class: Gold
priority: 3
type: reg
weight: 100
DSCPs: 26

class: Silver
priority: 4
type: llq
weight: 100
DSCPs: 28

class: Bronze
priority: 5
type: llq
weight: 200
DSCPs: default

class: Copper
priority: 15
type: reg
weight: 50
DSCPs: 10,12,14

Statistics example shows statistics for the previously defined classes:

class   priority type     weight     rx         tx        drops

Diamond   1       llq      0         2775        2650      0

Platinum  2       llq      0         1024        1020      105

Gold      3       reg      100       1775015     1773805   205

Silver    4       reg      100       1862437     1862336   550

Bronze    5       reg      200       3370033     2955120   3147

Copper    15      reg      50        1862437     762336    100689

From this statistical output, it is apparent that:

Monitoring Memory Resources

Use the fw vsx mstat command to monitor the memory the VSX Gateway uses. The command shows an overview of the memory that the system and each virtual device is using. These are the global memory resources that are shown:

The virtual devices are listed according to the VSIDs. Run vsx stat -v to show the VSID for the virtual devices.

You must be in expert mode to run the fw vsx mstat command. After you enable memory resource monitoring, it is necessary to reboot the VSX Gateway to use the feature.

Managing fw vsx mstat

Use the fw vsx mstat command to enable or disable memory resource monitoring on the VSX Gateway.

Syntax

fw vsx mstat {enable|disable|status}

Parameter

Description

enable

Enables memory resource monitoring. Reboot the VSX Gateway.

disable

Disables memory resource monitoring.

status

Shows if memory resource monitoring is enabled or disabled.

Example

fw vsx mstat disable

Output

VSX memory resource control status: disabled

Memory Resources for Each virtual device

Use the fw vsx mstat command to show the memory that each virtual device uses. Use the -vs parameter to show only some of the virtual devices.

You can also use these parameters for more data:

Syntax

fw vsx mstat [-vs <VSID>] [unit <measure>] [sort <top>]

Parameter

Description

-vs

Shows the memory usage of the specified virtual devices.

<VSID>

The ID of the virtual device.

To show multiple devices:

  • Put a space between each VSID: -vs 1 3 5
  • List a range of VSIDs: -vs 1-4

Note: You can combine VSID ranges together with single VSIDs

unit

Change the memory measurement unit shown in the command output.

<measure>

The memory measurement unit. The default value is megabytes.

The values are:

  • B - bytes
  • K, KB - kilobytes
  • M, MB - megabytes (default)
  • G, GB - gigabytes

sort

Sort the results according to the virtual devices that use the most memory.

<top>

Enter the maximum number of virtual devices to be shown. Only those virtual devices that use the most memory are shown.
Use all to show all virtual devices.

Example

fw vsx mstat -vs  0 1 3 5-8 unit MB sort 5
fw vsx mstat sort 5

Output (Both examples show the same results)

VSX Memory Status
=================
Memory Total: 997.22 MB
Memory Free: 232.56 MB
Swap Total: 2047.34 MB
Swap Free: 2047.16 MB
Swap-in rate: 0.00 MB
 VSID | Memory Consumption 
======+====================
    0 |          133.50 MB
    8 |           92.41 MB
    3 |           43.81 MB
    6 |           42.47 MB
    1 |           42.47 MB

Configuring Swap-in Sample Rate

The swap-in rate measures how much memory per second that the system swaps-in from disk. You can configure how often the system calculates the swap-in rate. For example, a sample rate of 5 means that the system calculates the swap-in rate every five minutes.

Syntax

fw vsx mstat swap <minutes>

Parameter

Description

swap

Configures the swap-in sample rate for the system.

<minutes>

Number of minutes that the system measures memory swaps to determine the swap-in rate. Only integers are valid values.

The default swap-in sample rate is 10.

Example

fw vsx mstat swap 5

Output

Swap-in sample rate was changed successfully to 5 minutes.

Comments

Swap-in sample rate is a system wide Linux setting. When you change the value for memory monitoring, all the swap-in rates are calculated according to the new value.

When you enable the monitoring memory resources feature, the swap-in rate setting is saved. When you disable the feature, the system restores the saved setting.

Using Debug Mode

Use the debug parameter to show more data about the memory that the VSX Gateway uses. You cannot use the -vs, unit and sort parameters in debug mode. The memory is shown in kilobytes.

Syntax

fw vsx mstat debug

Output

VSX Memory Status
=================
Memory Total: 1021152.00 KB
Memory Free: 324788.00 KB
Swap Total: 2096472.00 KB
Swap Free: 2096404.00 KB
Swap-in rate: 375.34 KB
 
 VSID |      Private_Clean |      Private_Dirty |    DispatcherGConn
======+====================+====================+====================+
    0 |        13544.00 KB |       144268.00 KB |            0.00 KB |
    1 |         1740.00 KB |        46276.00 KB |            0.00 KB |
    2 |         1720.00 KB |        46868.00 KB |            0.00 KB |
    3 |         1720.00 KB |        46644.00 KB |            0.00 KB |
    4 |         1712.00 KB |        45144.00 KB |            0.00 KB |
    5 |         1712.00 KB |        45836.00 KB |            0.00 KB |
    6 |         1720.00 KB |        45000.00 KB |            0.00 KB |
    7 |         1720.00 KB |        45044.00 KB |            0.00 KB |

Comments

By default the debug parameter shows these memory fields:

Private_Clean - Clean private pages. (/proc/[pid]/smaps)

Private_Dirty - Dirty private pages. (/proc/[pid]/smaps)

DispatcherHTab - Hash table for each Virtual System.

DispatcherGConn -Global connections for each Virtual System.

SecureXL - SecureXL memory each Virtual System uses.

VSX CPU Monitoring Commands

Use the fw vsx resctrl commands to monitor the CPU resources on a VSX Gateway. You can also see real-time statistics on the current and average CPU consumption by the virtual devices.

fw vsx resctrl load_configuration

Description

Initializes Resource Control. This command uses the contents of the resctrl file to configure CPU resource Control.

Syntax

fw vsx resctrl load_configuration

Output

Loading Resource Control configuration from $FWDIR/conf/resctrl
 Resource Control Monitor is disabled
 
 Resource Control monitoring configuration successfully read.

Comment

After you run fw vsx resctrl load_configuration, the commands in the resctrl file enable or disable Resource Control monitoring.

fw vsx resctrl monitor

Description

Configures the Resource Monitor and shows its current status. This command overrides the settings in the Resource Control configuration file, but does not survive reboot.

Syntax

fw vsx resctrl monitor {enable | disable | show}

Parameter

Description

enable

Enables Resource Monitor

disable

Disables Resource Monitor

show

Displays whether Resource Monitor is enabled or disabled

Example

fw vsx resctrl monitor enable

Output

Resource Control Monitor is enabled

fw vsx resctrl reset

Description

Resets the Resource Control monitoring statistics.

Syntax

fw vsx resctrl reset

fw vsx resctrl stat

Description

Shows the percentage of the total CPU resources that each virtual device uses.

Syntax

fw vsx resctrl [-u | -d| -d -q] stat

Parameter

Description

-u

Displays information per CPU (SMP only).

-d

Displays 24 hours of statistics. These statistics are only available after 24 hours of monitoring.

-d -q

Same as -d parameter without details for each CPU. Shows an average of all the CPUs for each virtual device.

Example

fw vsx resctrl -d -q stat

Output

Virtual Systems CPU Usage Statistics
====================================
Number of CPUs/Hyper-threading: 4
Monitoring active time: 14s
ID Name                      |1sec  10sec 1min  1hr   24hr*
=============================+====================================
0 VSX2                       |0.11  0.06  0.08  0.07  0.01
1 VSX2_vs1                   |15.80 21.57 21.75 22.28 1.94
2 VSX2_vsw                   |0.00  0.00  0.00  0.00  0.00
3 VSX2_vs2                   |16.91 22.57 22.77 23.09 2.01
=============================+====================================
Total Virtual Devices CPU Use|32.82 44.20 44.60 45.44 3.96
=============================+====================================

Sample Output fw vsx resctrl -u

Virtual Systems CPU Usage Statistics [%]
========================================
 
Number of CPUs: 4
Monitoring active time: 9m 4s
 
ID    Name                   | CPU  |  1sec  10sec   1min    1hr*  24hr*
=============================+======+==================================
  0   vsxb                   | 0    |  0.30   0.44   0.48   0.00   0.00
                             | 1    | 12.50   1.33   0.48   0.00   0.00
                             | 2    |  3.40   1.36   5.25   0.00   0.00
                             | 3    |  0.20   0.24   0.41   0.00   0.00
                             | Avg. |  4.10   0.84   1.65   0.00   0.00
-----------------------------+------+----------------------------------
  1   vs1                    | 0    |  0.00   0.02   0.11   0.00   0.00
                             | 1    |  0.00   0.01   0.01   0.00   0.00
                             | 2    |  0.20   0.04   0.11   0.00   0.00
                             | 3    |  0.00   0.01   0.01   0.00   0.00
                             | Avg. |  0.05   0.02   0.06   0.00   0.00
=============================+======+==================================
Total Virtual Devices CPU Use| 0    |  0.30   0.46   0.59   0.00   0.00
                             | 1    | 12.50   1.34   0.48   0.00   0.00
                             | 2    |  3.60   1.40   5.35   0.00   0.00
                             | 3    |  0.20   0.25   0.42   0.00   0.00
                             | Avg. |  4.15   0.86   1.71   0.00   0.00
=============================+======+==================================
 

Comments

SNMP Monitoring

You can only send SNMP traps to VS0.

For more about using SNMP, see SNMP in the R76 Gaia Administration Guide.

VSX SNMP Modes

There are two modes of SNMP monitoring that you can use with VSX:

VS mode lets you monitor all of the Virtual Systems in the VSX Gateway. Default mode only monitors VS0.

Note - When SNMP VS mode is enabled, traps from Virtual Systems are not supported.

Supported SNMP Versions

The VS mode uses SNMP version 3 to query the Virtual Systems. You can run remote SNMP queries on Virtual Systems in the VSX Gateway.

For systems that only support SNMP versions 1 and 2:

Enabling VS Mode

To enable VS mode on the VSX Gateway:

  1. Create an SNMP v3 user.
  2. Enable VS mode.
  3. Start the SNMP agent.

Sample commands:

> add snmp usm user admin security-level authNoPriv auth-pass-phrase abcd1234
> set snmp mode vs
> set snmp agent on

Note - If you run SNMP in the VS mode, you must run the snmpwalk command and set the -n switch to ctxname_vsid<vsid>.

Example:

snmpwalk -n ctxname_vsid2 -v 3 -l authNoPriv -u admin -A abcd1234 192.2.2.2 ifDesc

Sample SNMP VS Mode Configuration

Sample remote query of interfaces for VS2:

> snmpwalk -n ctxname_vsid2 -v 3 -l authNoPriv -u admin -A abdc1234 ifDescr

Sample local query of interfaces for VS2:

> vsenv 2
> snmpwalk -v 2c -c public localhost ifDescr

Note - You must enable VSX Resource Control Monitoring in order to see information about CPU usage per Virtual System.

Configuring Jumbo Frames

VSX supports Jumbo Frames and lets you configure up to the maximum MTU of the NIC driver.

Jumbo Frames on a Virtual System

Configure a Virtual System to enable Jumbo Frames on these interfaces types:

When you configure the MTU of a bond interface, the MTU of all the slave interfaces are automatically changed to the new value.

Configuring the MTU on Warp Links

Configuring the MTU on VLANs

To configure Jumbo Frames on a Virtual System:

  1. Open SmartDashboard.
  2. From the Network Objects tree, right-click the Virtual System and select Edit.

    The General Properties window opens.

  3. Click Topology.
  4. Select the interface and click Edit.

    The Interface Properties window opens.

  5. Configure the MTU for the interface.
  6. Click OK.

Jumbo Frames on a Virtual Switch

Configure the MTU of a Virtual Switch to enable Jumbo Frames on the Virtual Systems that are connected to the Virtual Switch. When you configure the MTU of the Virtual Switch, all the related Warp Links and interfaces are automatically changed to the new value.

You cannot configure the MTU of a Warp Link from the Virtual System.

To configure Jumbo Frames on a Virtual Switch:

  1. Open SmartDashboard.
  2. From the Network Objects tree, right-click the Virtual Switch and select Edit.

    The General Properties window opens.

  3. Click Topology.
  4. Configure the MTU for the Virtual Switch.
  5. Click OK.

Jumbo Frames on a Virtual Router

Configure the MTU of a Virtual Router to enable Jumbo Frames on the interfaces. To change the MTU of Warp Links, configure the settings of the Virtual System.

To configure Jumbo Frames on a Virtual Router:

  1. Open SmartDashboard.
  2. From the Network Objects tree, right-click the Virtual Router and select Edit.

    The General Properties window opens.

  3. Click Topology.
  4. Select the interface and click Edit.

    The Interface Properties window opens.

  5. Configure the MTU for the interface.
  6. Click OK.

Jumbo Frames on a Virtual System in Bridge Mode

Configure the MTU of a Virtual System in Bridge Mode to enable Jumbo Frames on the interfaces.

To configure Jumbo Frames on a Virtual System in Bridge Mode:

  1. Open SmartDashboard.
  2. From the Network Objects tree, right-click the Virtual System and select Edit.

    The General Properties window opens.

  3. Click Topology.
  4. Configure the MTU for the interface.
  5. Click OK.