Filter Configuration

Overview of the Filtering Feature

The Log Exporter can filter logs based on the field values.

Because field mapping operation is done before the actual filtering, make sure to use the dst name / value of the fields, if configured specifically in the <Format>FieldsMapping.xml file (based on the format). Otherwise, use the name / value as written in the raw log file.

It is possible to configure what to export or what not to export.

The filter configuration file is located in this file:

$EXPORTERDIR/targets/<Name of Log Exporter Configuration>/conf/FilterConfiguration.xml

Parameter

Description

Valid / Default Values

 
<filterGroup operator=""></filterGroup>

A group of fields that determine what to export.

There can be only one such parameter in the FilterConfiguration.xml file.

The relation between the fields is determined by the operator value.

See Mapping of Log Fields for Advanced Fields Configuration.

operator[and | or] 

<field name="" operator="">
  <value operation=""></value>
</field>

Declare a single field filter that participates in the filter group.

  • name

    The name of the field to filter on.

  • operator

    Declares the operator (and / or) between the various declared operations.

  • operation

    Declares the matching logic regarding the declared value.

  • value

    The specific value to filter on. Multiple values for a single operation is supported and should be added as a separate row.

operator[and | or]

 

operation[eq - equal | neq - not equal | gt - greater than | lt - less than ]

Configuring the Filtering Feature

These are the ways to configure the filtering feature:

  • This command configures filtering only for the fields Action, Blade, and Origin.

    The syntax must not contain spaces between the values:

    cp_log_export set name <name> filter-action-in "value1,value2"

    cp_log_export set name <name> filter-origin-in "value1,value2"

    cp_log_export set name <name> filter-blade-in "value1,value2"

    In addition, it is possible to use predefined families for "filter-blade-in" value:

    • Access

      For exporting Access logs only:

      • Application Control

      • Compliance

      • Connectra

      • Content Awareness

      • Core

      • DDoS Protector

      • Firewall

      • Identity Awareness

      • Identity Logging

      • Mobile Access

      • Security Gateway/Management

      • UA WebAccess

      • URL Filtering

      • VPN-1 & FireWall-1

    • TP

      For exporting Threat Prevention logs only:

      • Anti-Bot

      • Anti-Malware

      • Threat Emulation

      • IPS

      • IPS-1

      • SmartDefense

      • Anti-Virus

      • New Anti-Virus

      • Anti-Spam and Email Security

      • Threat Extraction

      • MTA

    • EndPoint

      For exporting Endpoint logs only:

      • Anti-Bot

      • Anti-Malware

      • Threat Emulation

      • IPS

      • IPS-1

      • SmartDefense

      • Anti-Virus

      • New Anti-Virus

      • Anti-Spam and Email Security

      • Threat Extraction

      • MTA

    • Mobile

      For exporting Mobile logs only:

      • WIFI Network

      • Mobile App

      • OS Exploits

      • Device

      • Network Security

      • Cellular Network

      • Network Access

      • iOS Profiles

      • Text Message

      • On-device Network Protection

    Example:

    cp_log_export set name <name> filter-blade-in Access,TP,EndPoint

  • You can add new fields to the FilterConfiguration.xml file.

    For example:

    <filters>
  <filterGroup operator="and">
    <field name="action" operator="and">
    </field>
    <field name="origin" operator="and">
    </field>
    <field name="product" operator="or">
    </field>
    <field name="severity" operator="or">
      <value operation="eq">3</value>
      <value operation="eq">4</value>
    </field>
    <field name="service_id" operator="or"> 
        <value operation="eq">https</value>
        <value operation="eq">http</value>
    </field>
    <field name="src" operator="or"> 
        <value operation="eq">1.2.1.2</value>
        <value operation="eq">3.2.3.2</value>
    </field>
    <field name="dst" operator="or"> 
        <value operation="eq">1.2.1.2</value>
        <value operation="eq">3.2.3.2</value>
    </field>
    <field name="rule_uid" operator="or"> 
        <value operation="eq">a24163d2-f2f2-1426-52d1-fa42133d04bd</value>
        <value operation="eq">b54152f1-a3c2-7242-52c1-dc86434a28ac</value>
    </field>
    <field name="NAT_rule_uid" operator="and"> 
        <value operation="eq">a24163d2-f2f2-1426-52d1-fa42133d04ac</value>
    </field>
    </filterGroup>
</filters>

Limitations of the Filtering Feature

  • The relation between the values of the same operation is only logical "OR".

    Example:

    cp_log_export set name <target-name> filter-action-in "accept,drop"

    Only logs with 'action = "accept"' OR 'action = "drop"' are exported.

  • Filtering is not supported for any of these fields:

    • app_category

    • app_desc

    • app_properties

    • app_risk

    • app_rule_name

    • appi_name

    • category

    • cvpn_category

    • cvpn_resource

    • desc

    • HTTPS_inspection_rule_name

    • matched_category

    • name

    • properties

    • time

    • UUID

  • Filtering for a certain field with the a double "NOT" condition "not equal(value1) OR not equal(value2)" is not supported.

    When editing the filtering XML, make sure to have a maximum of one line of "neq" operation in each field.