SD-WAN Best Practices

This section describes different best practices for SD-WAN configuration.

VPN - Link Selection

Best Practice - Always configure the Link Selection in the Security Gateway object. If the SD-WAN service on the Security Gateway goes down for some reason, this makes sure the Security Gateway uses the applicable interface for Site-to-Site VPN.

  • When two or more Security Gateways with enabled SD-WAN participate in the same Site-to-Site VPN Community, these Security Gateways automatically initiate Site-to-Site VPN tunnels between their SD-WAN interfaces.

    These Security Gateways ignore the configuration in SmartConsole > Security Gateway object > IPsec VPN > Link Selection, and select the interfaces based on the WAN Link Mapping configuration.

  • When a Security Gateway with enabled SD-WAN participate in the same Site-to-Site VPN Community with a Security Gateway without SD-WAN, these Security Gateways use the configuration in SmartConsole > Security Gateway object > IPsec VPN > Link Selection.

SD-WAN Policy - Rule Matching

Best Practices:

  1. Make sure that:

    • Traffic that should go through SD-WAN, matches the correct SD-WAN Policy rule and behavior.

    • Traffic that should not go through SD-WAN, does not match a "Local Breakout" rule, so it goes through the operating system routing.

  2. In a rule with the "Local Breakout" behavior:

    • Do not use "Any" in the Destination column.

    • Do not use any object that can match non-Internet traffic (inter-VLAN communication, traffic to next hop gateways, traffic in DMZ networks, and so on) in the Destination column because the Security Gateway would send the matched non-Internet traffic directly to its ISPs.

  3. In a rule with the "Backhaul" behavior:

    • Do not use "Any" in the Destination column.

    • Do not use any object that can match non-Internet traffic (inter-VLAN communication, traffic to next hop gateways, traffic in DMZ networks, and so on) in the Destination column because the Security Gateway would send the matched non-Internet traffic directly to its ISPs.

  4. Configure rules with the "Overlay - VPN" behavior above rules with the "Local Breakout" behavior.

  5. Do not use the "Internet" object in rules.

    This object cannot differentiate between an actual Internet traffic, "Overlay - VPN" traffic, or a cleartext traffic that is routed through an MPLS next hop router.

Note - The Security Gateway always matches the original connection direction.

The Security Gateway matches the first applicable rule in SD-WAN Policy and stops processing the rules.

The SD-WAN Policy rules that use the Breakout behavior work similar to Policy-Based Routing (PBR). During the routing process, the Security Gateway first applies these rules and overrides the operating system routes when these rules match (including the directly connected networks).

SD-WAN Rule

Guidelines and Security Gateway Behavior

Rule with the "Overlay - VPN" behavior

  • If the Security Gateway with enabled SD-WAN has to encrypt the traffic, and the VPN peer is also a Security Gateway with enabled SD-WAN, then the Security Gateway routes the traffic based on the rule steering behavior.

  • If the Security Gateway with enabled SD-WAN has to encrypt the traffic, and the VPN peer is a Security Gateway without enabled SD-WAN, then the Security Gateway routes the traffic based on the configured Link Selection in the Security Gateway object.

  • If the Security Gateway with enabled SD-WAN does not have to encrypt the traffic, the Security Gateway routes the traffic based on the operating system routing table.

Rule with the "Local Breakout" behavior

  • Make sure the Security Gateway does not match the "Overlay - VPN" traffic to rules with the "Local Breakout" behavior.

    Otherwise, the Security Gateway cannot use all VPN Transports / WAN Links to the SD-WAN VPN peer to forward the traffic.

  • The Security Gateway sends the cleartext traffic to its ISPs based on your steering configuration.

    Make sure the Security Gateway matches only the applicable outbound Internet traffic to rules with the "Local Breakout" behavior.

    Make sure the Security Gateway does not match cleartext non-Internet traffic (inter-VLAN communication, traffic to next hop gateways, traffic in DMZ networks, and so on).

Rule with the "Backhaul" behavior

  • The Branch Security Gateway encrypts the traffic and sends it to the Center Security Gateway over a Site-to-Site VPN tunnel.

    The Center Security Gateway sends this traffic to the Internet as a "Local Breakout".

  • The Center Security Gateway does not send the traffic to the Internet, if the traffic is designated the Center Security Gateway. This traffic behaves as "Overlay - VPN".

SD-WAN Probing

SD-WAN Overlay Probing is a mechanism to measure the SLA over SD-WAN Overlay Transports (the channels used to carry data between the SD-WAN interfaces on the local Security Gateway and the SD-WAN interfaces on the remote VPN peer.

The SD-WAN Overlay Probing method uses ICMP Echo Request packets that each SD-WAN interface on the local Security Gateway sends to each corresponding SD-WAN interface on the remote VPN peer.

Best Practice - Probing over an Overlay VPN tunnel must match an SD-WAN rule that contains the "Overlay - VPN" behavior. In such a rule, in the Source column and in the Destination column, you must include all real IP addresses of all SD-WAN interfaces of all SD-WAN Security Gateways (it is not necessary to include NATed IP addresses).

The Security Gateway sends the ICMP Echo Requests to its SD-WAN peer regardless of the SD-WAN Policy rules - the Security Gateway automatically sends this probing traffic from the applicable VPN interface.

The Security Gateway receives the ICMP Echo Replies from its SD-WAN peer based on the SD-WAN Policy rules.

Therefore, this asymmetric scenario is possible:

  1. The Security Gateway "GW1" sends the ICMP Echo Requests through "ISP1" of "GW1" to the peer Security Gateway "GW2" through "ISP1" of "GW2".

  2. The Security Gateway "GW2" sends the ICMP Echo Replies through "ISP2" of "GW2" to the Security Gateway "GW1" through "ISP1" of "GW1".

The SD-WAN Wizard creates these default SD-WAN Policy rules:

Source

Destination

Services & Applications

Behavior

Any

My VPN Domain

 

Peer VPN Domain

Any

Overlay - VPN

Any

SD-WAN Internet

Any

Default breakout

For more information about these three Dynamic Objects, see Dynamic Objects in SD-WAN Policy.

  • The purpose of the top rule with the Dynamic Objects "My VPN Domain" and "Peer VPN Domain" is also to match all encrypted traffic to all private IP addresses (for example, MPLS interfaces) of all Security Gateways and to apply the "Overlay - VPN" behavior to this traffic.

  • The purpose of the bottom rule with the Dynamic Object "SD-WAN Internet" is also to match all encrypted traffic to the public IP addresses of all Security Gateways and to apply the "Overlay - VPN" behavior to this traffic.

    The Steering Behavior object "Default breakout" has the routing preference "Prioritize Local Breakout", which equals to having both "Local Breakout" and "Backhaul". "Backhaul" between two Security Gateways acts like "Overlay - VPN".

If it is necessary to change the default "Local Breakout" rule from the "Default breakout" behavior to the routing preference "Local Breakout Only", you must create this explicit rule at the top of the SD-WAN Policy:

Source

Destination

Services & Applications

Behavior

All your overlay

external IP addresses

of all Security Gateways

All your overlay

external IP addresses

of all Security Gateways

icmp-proto

Overlay - VPN

Note - If your Security Gateway runs:

Then you can use the Dynamic Object "My VPN Domain" and the Dynamic Object "Peer VPN Domain".

The Default Overlay rule that the SD-WAN Wizard creates with these Dynamic Objects matches the Overlay probing traffic automatically, without the need of the above explicit rule. See Dynamic Objects in SD-WAN Policy.

SD-WAN Next Hop Probing is a basic way of checking the ISP status in a fast manner, simply by probing the next hop of each SD-WAN interface.

SD-WAN Next Hop Probing can detect link failures and outages faster than the "Local Breakout" Probing. As a result, SD-WAN failover occurs faster.

After the next hop stops responding:

  1. The WAN Link or ISP associated with the corresponding SD-WAN interface is considered as "Down" immediately.

  2. The Security Gateway generates the event "ISP <X> DOWN".

  3. The "Local Breakout" Probing stop on the SD-WAN interface until the ISP state for this SD-WAN interface become "Up" again.

The ISP state changes to "Down" only in these cases:

  • The SD-WAN interface link state changed to "Down".

  • The next hop stopped responding, and the SD-WAN Next Hop Probing mechanism changed the ISP state.

The default SD-WAN Next Hop Probing:

Version

Default Next Hop Probing

By default, the Next Hop Probing uses ARP Requests.

If the next hop does not respond to ARP Requests, then the Next Hop Probing automatically falls back to ICMP Echo Requests.

By default, the Next Hop Probing uses ICMP Echo Requests.

There is no fall back to another probing protocol.

Notes:

  • When working with the SD-WAN Routing Preference "Prioritize Local Breakout", the traffic failover from the "Local Breakout" behavior to the "Backhaul" behavior occurs only after the state of each available ISP changes to "Down".

  • If the next hop of the Security Gateway does not respond reliably to probing, or if it is necessary to disable the Next Hop Probing, then ollow the procedure in the section SD-WAN Next Hop Probing on CloudGuard Network Security Gateways.

If your SD-WAN Security Gateway is a Virtual Machine in a cloud (a CloudGuard Network Security Gateway), and its next hop does not respond to ARP Requests or ICMP Echo Requests, then you must disable the SD-WAN ICMP Probing on this CloudGuard Network Security Gateway:

Important - In a Cluster, you must configure all the Cluster Members in the same way.

  1. Connect to the command line on the CloudGuard Network Security Gateway / each CloudGuard Network Cluster Member.

  2. If your default shell is Gaia Clish, then go to the Expert mode:

    expert

  3. If the file $FWDIR/conf/sdwan/sdwan_steering_params.json exists, create a backup copy.

    Otherwise, create this file.

    Run this long command:

    if [ -f $FWDIR/conf/sdwan/sdwan_steering_params.json ] ; then echo "Creating the backup ..." ; cp -v $FWDIR/conf/sdwan/sdwan_steering_params.json{,BKP} ; else echo "Creating the file ..." ; touch $FWDIR/conf/sdwan/sdwan_steering_params.json ; stat $FWDIR/conf/sdwan/sdwan_steering_params.json ; fi

  4. Edit the $FWDIR/conf/sdwan/sdwan_steering_params.json file:

    vi $FWDIR/conf/sdwan/sdwan_steering_params.json

  5. If this file is empty, then add all these lines.

    If this file already contains other parameters, then add this parameter on a new line:

    {
    "sdw_nexthop_probe_enabled" : 0
}

  6. Save the changes in the file and exit the editor.

  7. Restart the SD-WAN Steering process:

    sdwan_steering_stop ; sdwan_steering_start

Important - If your SD-WAN Security Gateway is a Virtual Machine in a cloud (a CloudGuard Network Security Gateway), and its next hop does not respond to ICMP Echo Requests, then you must disable the SD-WAN ICMP Probing on this CloudGuard Network Security Gateway:

In Quantum Spark Gateways, the SD-WAN next hop probing is disabled by default.

Follow these steps to enable it when necessary:

  1. Connect to the command line on the Quantum Spark Gateway / each Quantum Spark Cluster Member.

  2. Log in to the Expert mode.

  3. If the file $FWDIR/conf/sdwan/sdwan_steering_params.json exists, create a backup copy.

    Otherwise, create this file.

    Run this long command:

    if [ -f $FWDIR/conf/sdwan/sdwan_steering_params.json ] ; then echo "Creating the backup ..." ; cp -v $FWDIR/conf/sdwan/sdwan_steering_params.json{,BKP} ; else echo "Creating the file ..." ; touch $FWDIR/conf/sdwan/sdwan_steering_params.json ; stat $FWDIR/conf/sdwan/sdwan_steering_params.json ; fi

  4. Edit the $FWDIR/conf/sdwan/sdwan_steering_params.json file:

    vi $FWDIR/conf/sdwan/sdwan_steering_params.json

  5. If this file is empty, then add all these lines.

    If this file already contains other parameters, then add this parameter on a new line:

    {
    "sdw_nexthop_probe_enabled" : 1
}

  6. Save the changes in the file and exit the editor.

  7. Restart the SD-WAN Steering process:

    sdwan_steering_stop ; sdwan_steering_start

Routes

Best Practice - If the Security Gateway has a private line (for example, MPLS), which is part of the overlay, then configure routes that send traffic for all the overlay destination networks to the MPLS next hop.

In this case, even if all ISP links are down, and no active default route exist in the operating system kernel, the specific route on the private MPLS makes sure that the "Overlay - VPN" traffic continues to work over the private MPLS SD-WAN interface. Without an active route to the destination, the packet is lost in the routing table, before the VPN encryption takes place.