Importing External Custom Intelligence Feeds in CLI

You can import threat indicator feeds from external sources directly on the Security Gateway Dedicated Check Point server that runs Check Point software to inspect traffic and enforce Security Policies for connected network resources..

After you import the feeds for the first time and install policy, the Security Gateway automatically pulls and enforces the indicator file each time the feed file is updated.

The Security Gateway imports the file over HTTP or HTTPS, or by reading from a local file or local directory.

Important - You must import the feed files on each Security Gateway and each Cluster Member Security Gateway that is part of a cluster. separately.

You can import indicator feeds in the CLI in these formats:

CSV in the Check Point format
Custom CSV in other formats
STIX Structured Threat Information eXpression™. A language that describes cyber threat information in a standardized and structured way. XML (STIX 1.0)
Snort version 2.9 or lower.

Feed's Resource

The Feed's resource for all formats can be one of these:

Resource

Description

Syntax Example

URL

HTTP or HTTPS.

Note - HTTPS resource with a self-signed certificate prompts for a user agreement to update the Trusted CA bundle.

You can skip the certificate verification by running this command in the Expert mode on the Security Gateway before you run the "ioc_feeds" command:

export EXT_IOC_NO_SSL_VALIDATION=1

ioc_feeds add --feed_name remote_feed --transport http --resource "http://10.0.0.1/my_feeds/stix_feed.xml"

Local File

Local File on the Security Gateway.

ioc_feeds add --feed_name local_feed --transport local_file --resource "/home/admin/my_feed.csv"

Local Directory

Local Directory on the Security Gateway that contains the applicable files in the correct feed format.

ioc_feeds add --feed_name local_feed --transport local_directory --resource "/home/admin/my_feed_folder"

'ioc_feeds' CLI Commands for Managing External Custom Intelligence Feeds

Use these "ioc_feeds" commands in the Expert mode on the Security Gateway to import and manage threat indicator files.

Commands

Command	Description	Syntax Example
`ioc_feeds -h`	Shows the built-in help.
`ioc_feeds push`	Pushes feeds now.	`ioc_feeds push`
`ioc_feeds show`	Shows all existing feeds.	`ioc_feeds show`
`ioc_feeds show --feed_name <Feed>`	Shows details for the specified feed.	`ioc_feeds show --feed_name local_feed`
`ioc_feeds show_interval`	Shows the fetching interval.	`ioc_feeds show_interval`
`ioc_feeds set_interval <sec>`	Configures the interval (in seconds) for fetching all feeds.	`ioc_feeds set_interval 1000`
`ioc_feeds show_scanning_mode`	Shows the statue of the scanning mode.	`ioc_feeds show_scanning_mode`
`ioc_feeds set_scanning_mode {on\| off}`	Enables (`on`) or disables (`off`) the scanning mode.	`ioc_feeds set_scanning_mode off`
`ioc_feeds add`	Adds a new feed. Mandatory parameters: `--feed_name <Feed>` Configures the feed name. `--transport {http \| https} --resource <URL>` Specifies a remote feed URL. `--transport local_file --resource <Absolute Path to Local File>` Specifies a local feed file. `--transport local_directory --resource <Absolute Path to Local Directory>` Specifies a local feed directory. Must be inside the `/home/` directory. Optional parameters: `--state {true \| false}` Configures the feed state - active (`true`, this is the default) or inactive (`false`). `--feed_action {Prevent \| Detect \| Ask}` Configures the feed action (default - `Prevent`) `--user_name <user>` Specifies the username for the feed source - a prompt for a password appears. `--proxy none` Specifies not to use any proxy when connecting to the feed source. If you do not specify the "`--no_proxy`" or the "`--proxy`" parameter, the tool uses the Security Gateway proxy. `--proxy <proxy server>:<proxy port>` Overrides the Security Gateway proxy when connecting to the feed source. If you do not specify the "`--no_proxy`" or the "`--proxy`" parameter, the tool uses the Security Gateway proxy. `--proxy_user_name <user>` Specifies the username for the proxy server - a prompt for a password appears. `--test true` Performs a dry run - fetches and parses the specified feed but does not save its configuration.	Example 1 - local file feed: `ioc_feeds add --feed_name local_feed --transport local_file --resource /home/admin/my_feed.csv` Example 2 - remote feed through a proxy: `ioc_feeds add --feed_name remote_feed --transport http --resource "http://10.0.0.1/my_feeds/stix_feed.xml" --proxy 192.168.22.33:8080 --state false -feed_action Detect --user_name admin@example.com` Example 3 - dry run for a remote feed: `ioc_feeds add --feed_name remote_stix_file --transport http --resource "http://www.public_indicators.com/ioc_stix_file.xml" --test true`
`ioc_feeds modify`	Modifies an existing feed. Values of the feed parameters that are not specified, stay as they were before.	`ioc_feeds modify --feed_name local_feed --state true`
`ioc_feeds delete`	Deletes existing specified feed. Mandatory parameter: `--feed_name <feed>` Specifies the feed name.	`ioc_feeds delete --feed_name local_feed`

CSV Check Point and STIX Formats

Each record in CSV Check Point format and the STIX XML (STIX 1.0) format must have these fields:

Fields

Field	Description	Valid Values	Value Criteria	Field Type
`UNIQ-NAME`	Name of the observable	Free text	Must be unique	Mandatory
`VALUE`	A valid value for the type of the observable	As provided in this table	Value of a parameter	Mandatory
`TYPE`	Type of the observable	`URL`	Any valid URL Not case-sensitive	Mandatory
		Domain	Any URL Domain
		IP	Standard IPv4 address
		`IP Range`	A range of valid IPv4 addresses, separated by a hyphen: `<IP1>-<IP2>`
		`MD5`	Any valid MD5 hash
		`SHA1`	Any valid SHA1 hash
		`SHA256`	Any valid SHA256 hash
		`Mail-subject`	Any non-empty text string
		`Mail-from`	Can be one of these: A single email address Example: `abc@domain.com` An email domain Examples: `@domain.com`, or `domain.com`
		`Mail-to`	Can be one of these: A single email address Example: `abc@domain.com` An email domain Examples: `@domain.com`, or `domain.com`
		`Mail-cc`	Can be one of these: A single email address Example: `abc@domain.com` An email domain Examples: `@domain.com`, or `domain.com`
		`Mail-reply-to`	Can be one of these: A single email address Example: `abc@domain.com` An email domain Examples: `@domain.com`, or `domain.com`
`CONFIDENCE`	Degree of confidence the observable presents	`low` `medium` `high` `critical`	Default - `high`	Optional
`SEVERITY`	Degree of threat the observable presents	`low` `medium` `high` `critical`	Default - `high`	Optional
`PRODUCT`	Check Point Software Blade Specific security solution (module): (1) On a Security Gateway, each Software Blade inspects specific characteristics of the traffic (2) On a Management Server, each Software Blade enables different management capabilities. that processes the observable	`AV` `AB`	`AV` - Check Point Anti-Virus Check Point Software Blade on a Security Gateway that uses real-time virus signatures and anomaly-based protections from ThreatCloud to detect and block malware at the Security Gateway before users are affected. Acronym: AV. Software Blade (default) Note - Only the Anti-Virus Software Blade can process MD5, SHA1, and SHA256 observables. `AB` - Check Point Anti-Bot Check Point Software Blade on a Security Gateway that blocks botnet behavior and communication to Command and Control (C&C) centers. Acronyms: AB, ABOT. Software Blade	Optional
`COMMENT`		Free text		Optional

Notes

If an optional field is empty, the default value is used.
If a mandatory field is empty, the Indicator Pattern of relevant observable malicious activity in an operational cyber domain, with relevant information on how to interpret it and how to handle it. file does not load.
STIX 2.0 (JSON file) is not supported.
Custom Indicators CLI (load_indicators) are not supported.

The supported STIX elements are:

stix:STIX_Package

stix:STIX_Header

stix:Title

stix:Description

stix:Indicators

stix:Indicator

indicator:Title

indicator:Type

indicator:Description

indicator:Observable Event or stateful property that can be observed in an operational cyber domain.

cybox:Object

cybox:Properties

FileObj:Hashes

cyboxCommon:Hash

cyboxCommon:Type

cyboxCommon:Simple_Hash_Value

stix:Observables

cybox:Observable

URIObj:Value

URIObject:Value

AddressObject:Address_Value

AddressObj:Address_Value

AddressObj:AddressObjectType

AddressObjet:AddressObjectType

cybox:Title

Condition Type Enum and Condition Application Enum support the values Equals and Any.

Example:

<cyboxCommon:Simple_Hash_Value condition="Equals" apply_condition="ANY">

Example of a CSV Indicator File in Check Point Format

#! DESCRIPTION = indi file,,,,,,

"#! REFERENCE = Indicator Bulletin; Feb 20, 2014",,,,,,

# FILE FORMAT:,,,,,,

"# All lines beginning ""#"" are comments",,,,,,

"# All lines beginning ""#!"" are metadata read by the SW",,,,,,

"# UNIQ-NAME,VALUE,TYPE,CONFIDENCE,SEVERITY,PRODUCT,COMMENT",,,,,,

observ1,8d9b6b8912a2ed175b77acd40cbe9a73,MD5,medium,medium,AV,FILENAME:WUC Invitation Letter Guests.doc

observ2,76700f862a0c241b8f4b754f76957bda,MD5,high,high,AV,FILENAME:essais~.swf| NOTE:FWS type Flash file

observ4,5dda6d1446b3cdb0bd4a3f0adb85d030ff59e975,SHA1,low,high,AV,file_name.pdf
observ5,db435a875be456f088f11c579aa52f30bc83cfff272cfad5a3f6f4de74de0654,SHA256,high,high,AV,file_name.doc

observ7,http://somemaliciousdomain.com/uploadfiles/upload/exp.swf?info= 789c333432d333b4d4b330d133b7b230b03000001b39033b&infosize=00840000 ,URL,high,high,AV,IPV4ADDR:196.168.25.25

observ8,svr01.passport.ServeUser.com,Dpmain,low,high,AB,TCP:80| IPV4ADDR:172.18.18.25|NOTE:Embedded EXE Remote C&C and Encoded Data

observ9,somemaliciousdomain2.com,Domain,,low,AV,TCP:8080|IPV4ADDR:172.22.14.10

observ10,http://www.bogusdomain.com/search?q=%24%2B%25&form=MOZSBR&pc= MOZI,URL,low,low,AB,IPV4ADDR:172.25.1.5

observ11,http://somebogussolution.com/register/card/log.asp?isnew=-1&LocalInfo= Microsoft%20Windows%20XP%20Service%20Pack%202&szHostName= ADAM-E512679EFD&tmp3=tmp3,URL,medium,,AB,

observ14,172.16.47.44,IP,high,medium,AB,TCP:8080

observ15,172.16.73.69,IP,medium,medium,AV,TCP:443|NOTE:Related to Flash exploitation

observ16,abc@def.com,mail-to,,high,AV,"NOTE:truncated; samples have appended to the subject the string ""PH000000NNNNNNN"" where NNNNNNN is a varying number"

observ34,stamdomain.com,domain,,,AB,

observ35,stamdomain.com,mail-from,high,medium,AV,

observ37,xyz.com,mail-from,medium,medium,AB,

observ38,@xyz.com,mail-from,medium,medium,AB,

observ39,a@xyz.com,mail-from,medium,medium,AB,

Example of a STIX 1.0 XML Indicator File

<stix:STIX_Package

    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

    xmlns:stix="http://stix.mitre.org/stix-1"

    xmlns:indicator="http://stix.mitre.org/Indicator-2"

    xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1"

    xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2"

    xmlns:cybox="http://cybox.mitre.org/cybox-2"

    xmlns:cyboxCommon="http://cybox.mitre.org/common-2"

    xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2"

    xmlns:example="http://example.com/"

    xsi:schemaLocation="

    http://stix.mitre.org/stix-1 ../stix_core.xsd

    http://stix.mitre.org/Indicator-2 ../indicator.xsd

    http://stix.mitre.org/default_vocabularies-1 ../stix_default_vocabularies.xsd

    http://cybox.mitre.org/objects#FileObject-2 ../cybox/objects/File_Object.xsd

    http://cybox.mitre.org/default_vocabularies-2 ../cybox/cybox_default_vocabularies.xsd"

    id="example:STIXPackage-ac823873-4c51-4dd1-936e-a39d40151cc3"

    version="1.0.1">

    <stix:STIX_Header>

        <stix:Title>Example file watchlist</stix:Title>

        <stix:Package_Intent xsi:type="stixVocabs:PackageIntentVocab-1.0">Indicators - Watchlist</stix:Package_Intent>

    </stix:STIX_Header>

    <stix:Indicators>

        <stix:Indicator xsi:type="indicator:IndicatorType" id="example:Indicator-611935aa-4db5-4b63-88ac-ac651634f09b">

            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.0">File Hash Watchlist</indicator:Type>

            <indicator:Description>Indicator that contains malicious file hashes.</indicator:Description>

            <indicator:Observable id="example:Observable-c9ca84dc-4542-4292-af54-3c5c914ccbbc">

                <cybox:Object id="example:Object-c670b175-bfa3-48e9-a218-aa7c55f1f884">

                    <cybox:Properties xsi:type="FileObj:FileObjectType">

                        <FileObj:Hashes>

                            <cyboxCommon:Hash>

                                <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0" condition="Equals">MD5</cyboxCommon:Type>

                                <cyboxCommon:Simple_Hash_Value condition="Equals" apply_condition="ANY">0522e955aaee70b102e843f14c13a92c##comma##0522e955aaee70b102e843f14c13a92d##comma##0522e955aaee70b102e843f14c13a92e</cyboxCommon:Simple_Hash_Value>

                            </cyboxCommon:Hash>

                        </FileObj:Hashes>

                    </cybox:Properties>

                </cybox:Object>

            </indicator:Observable>

        </stix:Indicator>

    </stix:Indicators>

</stix:STIX_Package>

Custom CSV Format

Custom Intelligence Feeds feature supports different kinds of CSV structure files.

Syntax Rules of Custom CSV files

The supported observables are:

Name, Value, Type, Confidence, Severity, Product, Comment.

Define the file's format, delimiter, and the comment lines to skip:

Use "--format" and specify your observables inside square brackets.

Use "--comment" for content to ignore in the original file.

Notes:

Content specified within the square brackets of "--format" is fetched from the original file.
Content inside the square brackets of "--comment" is ignored.

The Value and Type observables are mandatory.
The Value observable is specified based on its location in the original file: #<location_of_item>.

For example:

If the Value observable is in the 3rd place in your CSV row, enter:

--format [value:#3]
For all other observables, you can enter their location in the original file, or specify their value.

For example, if you want the value of the Type observable to be the domain specified in every CSV row, enter:

--format [type:domain]
When the feed's resource is a remote source (transport equals HTTP or HTTPS), every time the feed is fetched, it parses based on the format that was specified for this feed.

Examples

Original CSV file is a list of domains

# This list consists of High Level Sensitivity website URLs

# Columns (tab delimited):

# (1) site

#

Site

4kqd3hniqgptupi3p.k7oudl.top

stggsjv6mqiibmax.torshop.li

grrgelpetkavanis4.pv

52ou5k3t73ypjije.ie7t8k.top

ja:many.cu.ma

If you enter this command, the Security Gateway takes the domain specified in the first place of every row, and ignores anything that starts with # and the word Site.

ioc_feeds add --feed_name domain_list --transport https --resource "https://isc.sans.edu/feeds/suspiciousdomains_High.txt" --format [type:domain,value:1] --comment "#, Site"

This is the original CSV file

# category Descriptive tag name for this entry. For this report,

# the text sshpwauth will appear.

#

# A commented footer section shows an aggregate account of ASNs and

# addresses seen in the current report

#

3 | organization A | 18.30.10.26 | 2018-12-15 08:16:39 | sshpwauth

3 | organization B | 18.30.21.197 | 2018-12-28 17:43:41 | sshpwauth

17 | organization C | 128.46.80.71 | 2019-01-04 17:56:00 | sshpwauth

111| organization D | 128.197.31.119 | 2019-01-10 03:12: 18| sshpwauth

If you enter this command, the Security Gateway takes the IP address from the 3rd place in the row, takes the comment from the second place in the row, and ignores all content preceded by #:

ioc_feeds add --feed_name ip_list_with_spaces --transport local_file --resource "/home/admin/ioc/ip_list_with_spaces.txt" --format [value:#3,comment:#2,type:ip] --comment [#] --delimiter "|"

Snort Format

This feature provides an ability to load Intelligence feeds in Snort format. With the Snort format, you can enhance your overall threat detection capabilities and control over your security operation

Snort rules use signatures to define attacks. The name of the imported Snort protection is the value of the msg field in the original Snort rule Set of traffic parameters and other conditions in a Rule Base (Security Policy) that cause specified actions to be taken for a communication session.. The Snort feed size is limited to 3000 observables and 6000 rules in total. If the feed exceeds these limits, it is not loaded.

Check Point supports Snort 2.9 version and lower. Snort syntax is described in the official documentation at snort.org.

Snort Rule Syntax

<Action> <Protocol> <Source IP Address> <Source Port> <Direction> <Destination IP Address> <Destination Port> (msg:"<Text>"; <Keyword>:"<Option>";)

SNORT rules have two logical parts: Rule Header and Rule Options.

SNORT Rule Header:

<Action> <Protocol> <Address> <Port> <Direction> <Address> <Port>

SNORT Rule Options:

<keyword>:"<option>"

Example for a Snort file:

This Snort rule is designed to generate an alert whenever it detects HTTP traffic containing the string ".pdf", which indicates an attempt to transfer PDF files over HTTP, potentially for blocking or further investigation

alert tcp $EXTERNAL_NET any -> $HOME_NET 80 (msg:"Block pdf files over HTTP"; content:".pdf";)

Parameter	Description
`alert tcp`	Specifies the protocol to inspect, in this case, TCP.
`$EXTERNAL_NET any -> $HOME_NET 80`	Defines the traffic flow direction. It instructs Snort to look for traffic from any external network (`$EXTERNAL_NET`) to any port (`any`) on the local network (`$HOME_NET`) on port 80 (HTTP).
`(msg:"Block pdf files over HTTP"; content:".pdf";)`	The rule itself. It consists of a descriptive message enclosed in double quotes (`msg:"Block pdf files over HTTP"`) and a content match (`content:".pdf"`) looking for the string ".pdf" in the HTTP traffic.

Supported SNORT Syntax

These are the generally supported syntax components. There are some limitations (see Unsupported SNORT Syntax).

Syntax components

Keyword

Description

length

Specifies the original length of the content that is specified in a protected_content rule digest

pcre

Lets you write rules with Perl-compatible regular expressions.

Example:

alert tcp any any -> any 80 (content:"/foo.php?id="; pcre:"/\/foo.php?id=[0-9]{1,10}/iU";)

flowbits

Lets rules track states during a transport protocol session. Used in conjunction with conversation tracking from the Session preprocessor.

Example:

alert tcp $HTTP_SERVERS any -> $EXTERNAL_NET 21 (msg: "Does not match state in FTP path"; flow: established, to_server; content: "targetfile"; nocase; fast_pattern; flow bits: isset,INFTPPATH;no_match;)

byte_test

Tests a byte field for a specific value (with operator).

Example:

alert udp $EXTERNAL_NET any -> $HOME_NET 123 (msg: "Header length longer than maximum"; content: "length|3d|"; byte_test: 4, >, 1024, 1, relative;)

byte_jump

Lets you write rules that skip over specific portions of the length-encoded protocols and perform detection in very specific locations.

Example:

alert udp any any -> any 123 (msg: "Check for 0001 after 0123"; content: "|30 31 32 33|"; byte_jump: 4,4, relative; content: "|30 30 30 31|"; distance: 1; relative;)

isdataat

Verifies that the payload has data at a specified location.

Example:

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg: "\r\n\r\nHas 300 byte after"; flow: established, to_server; content: "|0a 0d 0a 0d|"; isdataat: 300,relative; sid:11111111;)

no_match

Does not block traffic even if the rule matches. Used with the "flow bits" key word to set a flag without performing a block.

Example:

Supported Content Keyword Modifiers: "nocase", "rawbytes", "depth", "offset:", "distance:", "within", "urilen"
Supported Threshold Rule Types - Threshold, Both (Limit is not supported.)
Supported Macros - HTTP_PORTS (Interpreted as 80 and 8080 ports.)

Note - Make sure that SNORT Rules with the same flowbits flag have the same content in the msg field. Otherwise, they will not be under the same protection.

Debugging:

The $FWDIR/log/SnortConvertor.elg file on the Management Server Check Point Single-Domain Security Management Server or a Multi-Domain Security Management Server. contains is updated with the debug messages from the last SnortConvertor run import of SNORT rules.

To find failed rule debugs in this file, search for: Failed to convert rule

Unsupported SNORT Syntax

These combinations of keywords and modifiers are implemented differently in the IPS Check Point Software Blade on a Security Gateway that inspects and analyzes packets and data for numerous types of risks (Intrusion Prevention System). blade as SNORT protection rules than in SNORT Rules.

Best Practice - Test them before activating them in a production environment.

With HTTP content or PCRE modifiers
- http_raw_uri content or "I" PCRE modifiers
- http_stat_msg content or "Y" PCRE modifiers
- http_stat_code content or "S" PCRE modifiers
Without HTTP content or PCRE modifiers
- Two or more uses of http_header content or "H" PCRE modifiers
- Two or more uses of http_raw_header content or "D" PCRE modifiers
With 'depth' or 'offset' content and HTTP content that is one of these on the same content keyword, or '^' (carret) in 'pcre' with one of these HTTP 'pcre' modifiers on the same 'pcre' keyword
- http_header content or "H" PCRE modifiers
- http_raw_header content or "D" PCRE modifiers
- http_stat_msg content or "Y" PCRE modifiers
- http_stat_code content or "S" PCRE modifiers
- http_uri content or "U" PCRE modifiers
Use of depth or offset content, or ^ (carrot) in PCRE, without any http content, and with destination ports that are not HTTP_PORTS macro
http_client_body content or "P" PCRE modifier
A PCRE keyword with {} (curly braces) quantifier
Use of both content and byte_test keywords
http_header content modifiers or "H" PCRE modifiers enforced only on raw http data (not decoded and normalized header data)
Use of the urilen keyword, except in a SNORT Rule that has only http_uri and "U" PCRE modifiers, or http_raw_uri content modifier and I PCRE modifiers:
- If the SNORT Rule has only http_uri content or "U" PCRE modifiers, the size will be of the decoded and normalized buffer.
- If the SNORT Rule has only http_raw_uri content or "I" PCRE modifiers, the size will be of the raw uri buffer.

SSL Services

In addition to the conventional metadata service options, Check Point supports additional keywords specifically for SSL traffic.

Snort rules for SSL traffic can be defined using the metadata keyword.

In the Snort rule options add:

metadata: service <SSL service>;

Example

alert tcp any any -> any 443 (msg:"Fake SSL Certificate"; content:"|08 e4 98 72 49 bc 45 07 48 a4 a7 81 33 cb f0 41 a3 51 00 33|"; metadata: service sslHello;)

Options for <ssl service>

Service	Description
`sslHello`	The sslHello service will search the Client Hello or Sever Hello depending on the flow.
`sslCertificate`	The sslCertificate service will search the Client Certificate or Sever Certificate depending on the flow.
`sslKeyx`	The sslKeyx service will search the Client Key Exchange or Sever Key Exchange depending on the flow.
`sslHeartbeat`	The sslHeartbeat will search the SSL heartbeats.
`sslCiphersuite`	The sslCiphersuite will search the Cipher Suite sent by the client.

When you use the sslHello, sslCertificate, or sslKeyx services, it is necessary to define a flow direction as either "flow: to_server" or "flow: from_server".

Note - These services and content modifiers are unique to Check Point and will not be supported by other SNORT engines.