Troubleshooting SD-WAN with Kernel Debug

For complete information about the kernel debug procedure, see the Security Gateway Guide for your version > Chapter "Kernel Debug".

  1. Connect to the command line on the Security Gateway.

  2. Log in to the Expert mode.

  3. Start the minimum kernel debug:

    fw ctl zdebug -m SDWANRB all | grep PROB

  4. Press the CTRL+C keys to stop the kernel debug.

Warning - Schedule a maintenance window. This procedure significantly increases the load on the CPU of your Security Gateway.

Important:

  • Always consult Check Point Support about the specific debug flags to troubleshoot your issue.

  • To minimize the CPU load, do not capture traffic during this debug procedure.

Procedure:

  1. Connect to the command line on the Security Gateway.

  2. Log in to the Expert mode.

  3. Reset the kernel debug options:

    fw ctl debug 0

  4. Reset the kernel debug filter:

    fw ctl set int simple_debug_filter_off 1

  5. Reset the SecureXL debug options:

    fwaccel dbg resetall

  6. Allocate the debug buffer:

    fw ctl debug -buf 8200

  7. Configure the required kernel debug flags:

    1. Enable the debug flags in the "FW" module:

      fw ctl debug -m fw + drop conn

    2. Enable the debug flags in the "SDWAN" module:

      fw ctl debug -m SDWAN all

    3. Enable the debug flags in the "SDWANRB" module:

      fw ctl debug -m SDWANRB all

    4. If the issue is related to an "Overlay" connection, then enable the debug flags in the "VPN" module:

      fw ctl debug -m VPN all

    5. Optional: Enable the debug flags in the "APPI" module to get more information:

      fw ctl debug -m APPI all

    6. Optional: Enable the debug flags in the "UP" module to get more information:

      fw ctl debug -m UP all

  8. Configure the required SecureXL debug flags:

    1. Enable the debug flags in the "DEFAULT" module:

      fwaccel dbg -m default all

    2. Enable the debug flags in the "SDWAN" module:

      fwaccel dbg -m sdwan all

    3. If the issue is related to an "Overlay" connection, enable the debug flags in the "VPN" module:

      fwaccel dbg -m vpn all

    4. Optional: Enable the debug flags in the "API" module to get more information:

      fwaccel dbg -m api all

    5. Optional: Only if your Security Gateway is not under a traffic load, enable the debug flags in the "PKT" module to get more information:

      fwaccel dbg -m pkt all

  9. Start the kernel debug:

    fw ctl kdebug -T -f > /var/log/kernel_debug.txt

  10. If the issue is related to an "Overlay" connection, then start the VPN debug in the user space as described in sk180488:

    • Syntax for R81.20 and higher (regardless of a Jumbo Hotfix Accumulator):

      vpn debug trunc ALL=5

    • Syntax for R81.10 (regardless of a Jumbo Hotfix Accumulator):

      vpn debug trunc ALL=5

      ike debug trunc

      ike debug on TDERROR_ALL_ALL=5

  11. Replicate the issue / wait for the issue to occur.

  12. Press the CTRL+C keys to stop the kernel debug and the SecureXL debug.

  13. Reset the kernel debug options:

    fw ctl debug 0

  14. Reset the SecureXL debug options:

    fwaccel dbg resetall

  15. If the issue is related to an "Overlay" connection, then stop the VPN debug in the user space as described in sk180488:

    vpn debug off

  16. Collect the debug output files:

    1. The kernel debug:

      /var/log/kernel_debug.txt

    2. The VPN user space debug:

      See the list of files in sk180488