Kernel Debug Procedure

Alternatively, use the Kernel Debug Procedure with Connection Life Cycle.

Important:

  • Kernel debug increases the load on the CPU of Security Group Members. Schedule a maintenance window.

  • You must run the applicable commands in the Expert mode on the applicable Security Group.

Step

Instructions

1

Connect to the command line on the applicable Security Group.

2

Log in to the Expert mode.

3

Reset the kernel debug options:

g_fw ctl debug 0

4

Reset the kernel debug filters:

g_fw ctl set int simple_debug_filter_off 1

5

Configure the applicable kernel debug filters.

See Kernel Debug Filters.

6

Allocate the kernel debug buffer for each CoreXL Firewall instance:

g_fw ctl debug -buf 8200

7

Make sure the kernel debug buffer was allocated:

g_fw ctl debug | grep buffer

8

Enable the applicable debug flags in the applicable kernel modules:

g_fw ctl debug -m <module> {all | + <flags>}

See Kernel Debug Modules and Debug Flags.

9

Examine the list of the debug flags that are enabled in the specified kernel modules:

g_fw ctl debug -m <module>

10

Start the kernel debug:

g_fw ctl kdebug -T -f > /var/log/kernel_debug.txt

11

Replicate the issue, or wait for the issue to occur.

12

Stop the kernel debug:

Press the CTRL+C keys.

13

Reset the kernel debug options:

g_fw ctl debug 0

14

Reset the kernel debug filters:

g_fw ctl set int simple_debug_filter_off 1

15

Analyze the debug output file on each Security Group Member:

/var/log/kernel_debug.txt

Example - Connection 192.168.20.30:<Any> --> 172.16.40.50:80 

[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug 0
Defaulting all kernel debugging options
Debug state was reset to default.
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl set int simple_debug_filter_off 1
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl set str simple_debug_filter_saddr_1 "192.168.20.30"
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl set str simple_debug_filter_daddr_2 "192.168.20.40"
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl set int simple_debug_filter_dport_1 80
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug -buf 8200
Initialized kernel debugging buffer to size 8192K
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug | grep buffer
Kernel debugging buffer size: 8192KB
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug -m fw + conn drop
Updated kernel's debug variable for module fw
Debug flags updated.
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug -m fw
Kernel debugging buffer size: 8192KB
Module: fw
Enabled Kernel debugging options: error warning conn drop
Messaging threshold set to type=Info freq=Common
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl kdebug -T -f > /var/log/kernel_debug.txt

... ... Replicate the issue, or wait for the issue to occur ... ...

... ... Press CTRL+C ... ... 

[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl debug 0
Defaulting all kernel debugging options
Debug state was reset to default.
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_fw ctl set int simple_debug_filter_off 1
[Expert@MyChassis-ch0x-0x:0]#
[Expert@MyChassis-ch0x-0x:0]# g_all ls -l /var/log/kernel_debug.txt
-rw-rw---- 1 admin root 1630619 Apr 12 19:49 /var/log/kernel_debug.txt
[Expert@MyChassis-ch0x-0x:0]#