Importing External Custom Intelligence Feeds in CLI

You can import threat indicator feeds from external sources directly on the Security GatewayClosed Dedicated Check Point server that runs Check Point software to inspect traffic and enforce Security Policies for connected network resources..

After you import the feeds for the first time and install policy, the Security Gateway automatically pulls and enforces the indicator file each time the feed file is updated.

The Security Gateway imports the file over HTTP or HTTPS, or by reading from a local file or local directory.

Important - You must import the feed files on each Security Gateway and each Cluster MemberClosed Security Gateway that is part of a cluster. separately.

You can import indicator feeds through the CLI in these formats:

The Feed's resource for all formats can be one of these:

Resource

Description

Syntax Example

URL

HTTP or HTTPS.

Note - HTTPS resource with a self-signed certificate prompts for a user agreement to update the Trusted CA bundle.

You can skip the certificate verification by running this command in the Expert mode on the Security Gateway before you run the "ioc_feeds" command:

export EXT_IOC_NO_SSL_VALIDATION=1

ioc_feeds add --feed_name remote_feed --transport http --resource "http://10.0.0.1/my_feeds/stix_feed.xml"

Local File

Local File on the Security Gateway.

ioc_feeds add --feed_name local_feed --transport local_file --resource "/home/admin/my_feed.csv"

Local Directory

Local Directory on the Security Gateway that contains the applicable files in the correct feed format.

ioc_feeds add --feed_name local_feed --transport local_directory --resource "/home/admin/my_feed_folder"

'ioc_feeds' CLI Commands for Managing External Custom Intelligence Feeds

Use these "ioc_feeds" commands in the Expert mode on the Security Gateway to import and manage threat indicator files.

Command

Description

Syntax Example

ioc_feeds -h

Shows the built-in help.

 

ioc_feeds push

Pushes feeds now.

ioc_feeds push

ioc_feeds show

Shows all existing feeds.

ioc_feeds show

ioc_feeds show --feed_name <Feed>

Shows details for the specified feed.

ioc_feeds show --feed_name local_feed

ioc_feeds show_interval

Shows the fetching interval.

ioc_feeds show_interval

ioc_feeds set_interval <sec>

Configures the interval (in seconds) for fetching all feeds.

ioc_feeds set_interval 1000

ioc_feeds show_scanning_mode

Shows the statue of the scanning mode.

ioc_feeds show_scanning_mode

ioc_feeds set_scanning_mode {on| off}

Enables (on) or disables (off) the scanning mode.

ioc_feeds set_scanning_mode off

ioc_feeds add

Adds a new feed.

Mandatory parameters:

  • --feed_name <Feed>

    Configures the feed name.

  • --transport {http | https} --resource <URL>

    Specifies a remote feed URL.

  • --transport local_file --resource <Absolute Path to Local File>

    Specifies a local feed file.

  • --transport local_directory --resource <Absolute Path to Local Directory>

    Specifies a local feed directory.

    Must be inside the /home/ directory.

Optional parameters:

  • --state {true | false}

    Configures the feed state - active (true, this is the default) or inactive (false).

  • --feed_action {Prevent | Detect | Ask}

    Configures the feed action (default - Prevent)

  • --user_name <user>

    Specifies the username for the feed source - a prompt for a password appears.

  • --proxy none

    Specifies not to use any proxy when connecting to the feed source.

    If you do not specify the "--no_proxy" or the "--proxy" parameter, the tool uses the Security Gateway proxy.

  • --proxy <proxy server>:<proxy port>

    Overrides the Security Gateway proxy when connecting to the feed source.

    If you do not specify the "--no_proxy" or the "--proxy" parameter, the tool uses the Security Gateway proxy.

  • --proxy_user_name <user>

    Specifies the username for the proxy server - a prompt for a password appears.

  • --test true

    Performs a dry run - fetches and parses the specified feed but does not save its configuration.

Example 1 - local file feed:

ioc_feeds add --feed_name local_feed --transport local_file --resource /home/admin/my_feed.csv

Example 2 - remote feed through a proxy:

ioc_feeds add --feed_name remote_feed --transport http --resource "http://10.0.0.1/my_feeds/stix_feed.xml" --proxy 192.168.22.33:8080 --state false -feed_action Detect --user_name admin@example.com

Example 3 - dry run for a remote feed:

ioc_feeds add --feed_name remote_stix_file --transport http --resource "http://www.public_indicators.com/ioc_stix_file.xml" --test true

ioc_feeds modify

Modifies an existing feed.

Values of the feed parameters that are not specified, stay as they were before.

ioc_feeds modify --feed_name local_feed --state true

ioc_feeds delete

Deletes existing specified feed.

Mandatory parameter:

  • --feed_name <feed>

    Specifies the feed name.

ioc_feeds delete --feed_name local_feed

CSV Check Point and STIX Formats

Each record in CSV Check Point format and the STIX XML (STIX 1.0) format must have these fields:

Field

Description

Valid Values

Value Criteria

Field Type

UNIQ-NAME

Name of the observable

Free text

Must be unique

Mandatory

VALUE

A valid value for the type of the observable

As provided in this table

Value of parameter

Mandatory

TYPE

Type of the observable

URL

Any valid URL

Not case-sensitive

Mandatory

Domain

Any URL Domain

IP

Standard IPv4 address

IP Range

A range of valid IPv4 addresses,

separated by a hyphen: <IP1>-<IP2>

MD5

Any valid MD5 hash

SHA1

Any valid SHA1 hash

SHA256

Any valid SHA256 hash

Mail-subject

Any non-empty text string

Mail-from

Can be one of these:

  • A single email address

    Example: abc@domain.com

  • An email domain

    Examples: @domain.com, or domain.com

Mail-to

Can be one of these:

  • A single email address

    Example: abc@domain.com

  • An email domain

    Examples: @domain.com, or domain.com

Mail-cc

Can be one of these:

  • A single email address

    Example: abc@domain.com

  • An email domain

    Examples: @domain.com, or domain.com

Mail-reply-to

Can be one of these:

  • A single email address

    Example: abc@domain.com

  • An email domain

    Examples: @domain.com, or domain.com

CONFIDENCE

Degree of confidence the observable presents

  • low

  • medium

  • high

  • critical

Default - high

Optional

SEVERITY

Degree of threat the observable presents

  • low

  • medium

  • high

  • critical

Default - high

Optional

PRODUCT

Check Point Software BladeClosed Specific security solution (module): (1) On a Security Gateway, each Software Blade inspects specific characteristics of the traffic (2) On a Management Server, each Software Blade enables different management capabilities. that processes the observable

  • AV

  • AB

Optional

COMMENT

 

Free text

 

Optional

  • Use commas to separate the fields in a record

  • Enter one record per line, or use '\n' to separate the records

  • If free text contains quotation marks, commas, or line breaks, it must be enclosed in quotation marks

  • To enclose part of free text in quotations, use double quotation marks: "<text>"

#! DESCRIPTION = indi file,,,,,,

"#! REFERENCE = Indicator Bulletin; Feb 20, 2014",,,,,,

# FILE FORMAT:,,,,,,

"# All lines beginning ""#"" are comments",,,,,,

"# All lines beginning ""#!"" are metadata read by the SW",,,,,,

"# UNIQ-NAME,VALUE,TYPE,CONFIDENCE,SEVERITY,PRODUCT,COMMENT",,,,,,

observ1,8d9b6b8912a2ed175b77acd40cbe9a73,MD5,medium,medium,AV,FILENAME:WUC
Invitation Letter Guests.doc

observ2,76700f862a0c241b8f4b754f76957bda,MD5,high,high,AV,FILENAME:essais~.swf|
NOTE:FWS type Flash file

observ4,5dda6d1446b3cdb0bd4a3f0adb85d030ff59e975,SHA1,low,high,AV,file_name.pdf
observ5,db435a875be456f088f11c579aa52f30bc83cfff272cfad5a3f6f4de74de0654,SHA256,high,high,AV,file_name.doc

observ7,http://somemaliciousdomain.com/uploadfiles/upload/exp.swf?info=
789c333432d333b4d4b330d133b7b230b03000001b39033b&infosize=00840000
,URL,high,high,AV,IPV4ADDR:196.168.25.25

observ8,svr01.passport.ServeUser.com,Dpmain,low,high,AB,TCP:80|
IPV4ADDR:172.18.18.25|NOTE:Embedded EXE Remote C&C and Encoded Data

observ9,somemaliciousdomain2.com,Domain,,low,AV,TCP:8080|IPV4ADDR:172.22.14.10

observ10,http://www.bogusdomain.com/search?q=%24%2B%25&form=MOZSBR&pc=
MOZI,URL,low,low,AB,IPV4ADDR:172.25.1.5

observ11,http://somebogussolution.com/register/card/log.asp?isnew=-1&LocalInfo=
Microsoft%20Windows%20XP%20Service%20Pack%202&szHostName=
ADAM-E512679EFD&tmp3=tmp3,URL,medium,,AB,

observ14,172.16.47.44,IP,high,medium,AB,TCP:8080

observ15,172.16.73.69,IP,medium,medium,AV,TCP:443|NOTE:Related to Flash
exploitation

observ16,abc@def.com,mail-to,,high,AV,"NOTE:truncated; samples have appended to
the subject the string ""PH000000NNNNNNN"" where NNNNNNN is a varying number"

observ34,stamdomain.com,domain,,,AB,

observ35,stamdomain.com,mail-from,high,medium,AV,

observ37,xyz.com,mail-from,medium,medium,AB,

observ38,@xyz.com,mail-from,medium,medium,AB,

observ39,a@xyz.com,mail-from,medium,medium,AB, 

<stix:STIX_Package
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:stix="http://stix.mitre.org/stix-1"
    xmlns:indicator="http://stix.mitre.org/Indicator-2"
    xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1"
    xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2"
    xmlns:cybox="http://cybox.mitre.org/cybox-2"
    xmlns:cyboxCommon="http://cybox.mitre.org/common-2"
    xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2"
    xmlns:example="http://example.com/"
    xsi:schemaLocation="
    http://stix.mitre.org/stix-1 ../stix_core.xsd
    http://stix.mitre.org/Indicator-2 ../indicator.xsd
    http://stix.mitre.org/default_vocabularies-1 ../stix_default_vocabularies.xsd
    http://cybox.mitre.org/objects#FileObject-2 ../cybox/objects/File_Object.xsd
    http://cybox.mitre.org/default_vocabularies-2 ../cybox/cybox_default_vocabularies.xsd"
    id="example:STIXPackage-ac823873-4c51-4dd1-936e-a39d40151cc3"
    version="1.0.1">
    <stix:STIX_Header>
        <stix:Title>Example file watchlist</stix:Title>
        <stix:Package_Intent xsi:type="stixVocabs:PackageIntentVocab-1.0">Indicators - Watchlist</stix:Package_Intent>
    </stix:STIX_Header>
    <stix:Indicators>
        <stix:Indicator xsi:type="indicator:IndicatorType" id="example:Indicator-611935aa-4db5-4b63-88ac-ac651634f09b">
            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.0">File Hash Watchlist</indicator:Type>
            <indicator:Description>Indicator that contains malicious file hashes.</indicator:Description>
            <indicator:Observable id="example:Observable-c9ca84dc-4542-4292-af54-3c5c914ccbbc">
                <cybox:Object id="example:Object-c670b175-bfa3-48e9-a218-aa7c55f1f884">
                    <cybox:Properties xsi:type="FileObj:FileObjectType">
                        <FileObj:Hashes>
                            <cyboxCommon:Hash>
                                <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0" condition="Equals">MD5</cyboxCommon:Type>
                                <cyboxCommon:Simple_Hash_Value condition="Equals" apply_condition="ANY">0522e955aaee70b102e843f14c13a92c##comma##0522e955aaee70b102e843f14c13a92d##comma##0522e955aaee70b102e843f14c13a92e</cyboxCommon:Simple_Hash_Value>
                            </cyboxCommon:Hash>
                        </FileObj:Hashes>
                    </cybox:Properties>
                </cybox:Object>
            </indicator:Observable>
        </stix:Indicator>
    </stix:Indicators>
</stix:STIX_Package>

Custom CSV Format

Custom Intelligence Feeds feature supports different kinds of CSV structure files.

  • The supported observables are:

    Name, Value, Type, Confidence, Severity, Product, Comment.

  • Define the file's format, delimiter, and the comment lines to skip:

    Use "--format" and specify your observables inside square brackets.

    Use "--comment" for content to ignore in the original file.

    Notes:

    • Content specified within the square brackets of "--format" is fetched from the original file.

    • Content inside the square brackets of "--comment" is ignored.

  • The Value and Type observables are mandatory.

  • The Value observable is specified based on its location in the original file: #<location_of_item>.

    For example:

    If the Value observable is in the 3rd place in your CSV row, enter:

    --format [value:#3]

  • For all other observables, you can enter their location in the original file, or specify their value.

    For example, if you want the value of the Type observable to be the domain specified in every CSV row, enter:

    --format [type:domain]

  • When the feed's resource is a remote source (transport equals HTTP or HTTPS), every time the feed is fetched, it parses based on the format that was specified for this feed.

Examples

# This list consists of High Level Sensitivity website URLs

# Columns (tab delimited):

# (1) site

#

Site

4kqd3hniqgptupi3p.k7oudl.top

stggsjv6mqiibmax.torshop.li

grrgelpetkavanis4.pv

52ou5k3t73ypjije.ie7t8k.top

ja:many.cu.ma

If you enter this command, the Security Gateway takes the domain specified in the first place of every row, and ignores anything that starts with # and the word Site.

ioc_feeds add --feed_name domain_list --transport https --resource "https://isc.sans.edu/feeds/suspiciousdomains_High.txt" --format [type:domain,value:1] --comment "#, Site"

# category Descriptive tag name for this entry. For this report,

# the text sshpwauth will appear.

#

# A commented footer section shows an aggregate account of ASNs and

# addresses seen in the current report

#

3 | organization A | 18.30.10.26 | 2018-12-15 08:16:39 | sshpwauth

3 | organization B | 18.30.21.197 | 2018-12-28 17:43:41 | sshpwauth

17 | organization C | 128.46.80.71 | 2019-01-04 17:56:00 | sshpwauth

111| organization D | 128.197.31.119 | 2019-01-10 03:12: 18| sshpwauth

If you enter this command, the Security Gateway takes the IP address from the 3rd place in the row, takes the comment from the second place in the row, and ignores all content preceded by #:

ioc_feeds add --feed_name ip_list_with_spaces --transport local_file --resource "/home/admin/ioc/ip_list_with_spaces.txt" --format [value:#3,comment:#2,type:ip] --comment [#] --delimiter "|"

To learn more about Custom Intelligence Feeds, see sk132193.