Skyline Configuration on Check Point Servers that run Gaia OS - Prometheus with Grafana

Note - This section provides the steps for the Prometheus Server and the Grafana Server.

For other monitoring tools, see Skyline Configuration on Check Point Servers that run Gaia OS - Other Monitoring Tools.

This section applies to these Check Point Servers:

  • Security Gateways / VSX Gateways.

  • ClusterXL Members.

    In a Cluster, you must configure all the Cluster Members in the same way.

  • Security Groups on Scalable Platforms (ElasticXL Cluster, Maestro, and Scalable Chassis).

  • Security Management Servers.

  • Multi-Domain Security Management Servers.

  • Multi-Domain Log Servers.

  • Log Servers.

  • SmartEvent Servers.

  • Endpoint Security Management Servers.

  • Endpoint Policy Servers.

Video Tutorial

Watch a brief video tutorial on how to install and configure Skyline:

Step 1 - Install the Prometheus Server

Note - Skip this step if you have already installed the Prometheus Server

Installing a Prometheus Server

To install a Prometheus Server on the external server, refer to the Prometheus installation instructions for the various platforms.

Prometheus Server Default URL

http://localhost:9090

Configuring a Prometheus Server

  1. Mandatory: On the Prometheus Server, enable its Remote Write Receiver to get metrics data from the Check Point Servers. Refer to these Prometheus instructions.

  2. Optional: Use TLS Encryption and Basic authentication to secure the connection between the Prometheus Server and the OpenTelemetry Collector.

    The Prometheus Server and OpenTelemetry Collector support Transport Layer Security (TLS) encryption for their connection. Refer to these Prometheus instructions.

    Check Point also requires you to enable basic authentication to make the security bi-directional, refer to these Prometheus instructions.

    TLS configuration has two main components:

    • A pair of a Key and a Certificate, used to encrypt your communication.

    • (Optional) Certificate Authority (CA) that you trust, used to verify and trust the certificate of the other endpoint with which you communicate. If the certificate of the other endpoint is unknown to the CA, the communication is rejected.

    You can create these certificates:

    • CA-signed certificates: You create a key and a certificate request, which is then signed by the CA.

    • Self-signed certificates: You create a key and a certificate that is signed by the user.

    The steps below describe a self-signed certificate.

    To configure TLS, you must create two pairs of a key and a certificate on the Prometheus Server:

    1. Create a self-signed certificate and a private key on the Prometheus Server:

      1. Create the file called openssl.conf with the template below.

        Enter the applicable information in the "[ dn ]" and "[ alt_names ]" sections.

        Important - The Prometheus Server and the OpenTelemetry Collector must have different hostnames and IP addresses.

        Template:

        [ req ]
default_bits       = 4096
default_md         = sha256
req_extensions     = v3_req
distinguished_name = dn
prompt = no

[ v3_req ]
subjectAltName = @alt_names

[ dn ]
C = <Country Name>
ST = <State Or Province Name>
L = <Locality>
O = <Organization>
OU = <Organization Unit>
CN = <Common Name>

[ alt_names ]
DNS = <HOSTNAME>
IP = <IP ADDRESS>

      2. Generate the key and certificate pair on any Gaia OS server (in the Expert mode):

        cpopenssl req -x509 -newkey rsa:4096 -nodes -config openssl.conf -keyout mykey.key -out mycert.crt -extensions v3_req

        This command creates two files in the current working directory:

        • mykey.key

        • mycert.crt

        Move these two files from the Gaia OS server to the Prometheus Server.

    2. Use the newly generated key and certificate file to configure TLS on the Prometheus Server in the web-config.yaml configuration file (you may need to create this file).

      Example of a web-config.yaml file:

      tls_server_config:
   key_file: /home/admin/mykey.key
   cert_file: /home/admin/mycert.crt

Step 2 - Install the Grafana Server

Note - Skip this step if you have already installed the Grafana Server.

Installing a Grafana Server

To install a Grafana Server on the external server, refer to the Grafana installation instructions.

Note - You can install the Grafana Server on the same server that contains your Prometheus Server instance.

Grafana Server Default URL

http://localhost:3000

Configuring a Grafana Server (connecting Data-source and Dashboards)

  1. Log in to Grafana and add a new Prometheus Data Source with the Prometheus Server you created.

    On the side panel, click Configuration > Data sources > click the Add data source button > select the Prometheus data source.

    Make sure to mark the Prometheus data source as Default.

    Example:

  2. Import the Check Point Grafana dashboard to start monitoring your environment.

    On the side panel, click Create > Import > upload the applicable JSON file for a dashboard.

    Example:

  3. You can modify the existing dashboards or create new dashboards according to your needs.

    To see a full description of all the data exposed by Skyline, see Skyline Metrics Repository.

    Example:

Step 3 - Install the OpenTelemetry Agent and OpenTelemetry Collector on the Check Point Server

On an online Check Point Server:

The OpenTelemetry Agent (OtlpAgent) and the OpenTelemetry Collector (CPotelcol) packages are installed automatically on all applicable Check Point Servers, if you allowed the Automatic Update downloads as described in sk94508.

On an offline Check Point Server:

The minimum required OpenTelemetry Agent and OpenTelemetry Collector packages are also installed as part of the Jumbo Hotfix Accumulator installation.

See sk178566 > section "Requirements".

Related offline update packages:

Step 4 - Configure the OpenTelemetry Collector on the Check Point Server to work with the Prometheus Server

Important:

  • Do not cut and paste the commands below. Enter them manually.

  • In a Cluster, you must configure all the Cluster Members in the same way.

  • In Management High Availability, we recommend to configure all the Management Servers in the same way.

  • On a Maestro Orchestrator (MHO):

    You can run the CLI command "sklnctl" (in the Expert mode) or run the Gaia REST API command (requires Gaia API v1.7 or higher).

    The CLI command runs on the Orchestrator and configures only the Orchestrator.

  • In a Scalable Platform Security Group:

    You can run the CLI command "g_all sklnctl" (in the Expert mode) or run the Gaia REST API command (requires Gaia API v1.7 or higher).

  • For the applicable Gaia REST API commands, refer to the Check Point Gaia API Reference (v1.7 and higher):

    • In Gaia API v1.8 and higher: Section "Diagnostics" > Section "OpenTelemetry.

    • In Gaia API v1.7: Section "OpenTelemetry".

This step provides two different procedures - for an on-premises Prometheus Server, and for a Prometheus Server in AWS.

  1. Prepare the required payload for the command:

    Notes:

    • Download the sample payload files from sk178566 > section "Downloads".

      Replace the placeholder strings "<...>" in the sample files with your actual strings.

    • In the "basic" section, configure the "username" and "password" attributes to your monitoring server's username / password.

      The "password" attribute supports only these characters:

      • uppercase letters (A-Z)
      • lowercase letters (a-z)
      • digits (0-9)
      • underscore (_)
      • comma (,)
      • period (.)
      • backslash (\)
      • slash (/)
      • hyphen (-)

    • In the "ca-public-key" section, configure the TLS settings with the CA certificate of your monitoring server (as PEM X509), or use the self-signed certificate you generated previously.

      Replace "<CERTIFICATE>" with the CA certificate of the monitoring server (PEX X509) - paste the entire string:

      -----BEGIN CERTIFICATE-----<BASE64_TEXT>-----END CERTIFICATE-----

    • In the "url" attribute, configure your monitoring server's IP address / URL.

      If you do not wish to use TLS encryption in labs or test environments, then make these changes in the payload:

      1. In the "url" attribute, change the URL protocol from "https://" to "http://".

      2. Remove the "client-auth" and "server-auth" attributes.

        Note - When you apply the payload, this warning appears: "it is recommended to have both client and server authentication").

    Example JSON payloads:

    {
    "enabled": true,
    "export-targets": {
        "add": [
            {
                "client-auth": {
                    "basic": {
                        "username": "<USERNAME>",
                        "password": "<PASSWORD>"
                    }
                },
                "enabled": true,
                "server-auth": {
                    "ca-public-key": {
                        "type": "PEM-X509",
                        "value": "<CERTIFICATE>"
                    }
                },
                "type": "prometheus-remote-write",
                "url": "https://<EXTERNAL_PROMETHEUS_IP_ADDRESS>:9090/api/v1/write"
            }
        ]
    }
}
    		 
    {
    "enabled": true,
    "export-targets": {
        "add": [
            {
                "enabled": true,
                "type": "prometheus-remote-write",
                "url": "http://<EXTERNAL_PROMETHEUS_IP_ADDRESS>:9090/api/v1/write"
            }
        ]
    }
}

  2. Run the configuration command to apply the payload - either the CLI command (in the Expert mode), or the Gaia REST API command:

    • Method 1 - Run the CLI command "sklnctl":

      1. Save the JSON payload in a file (for example, /home/admin/payload.json).

      2. On a Scalable Platform Security Group, copy the JSON payload file to all Security Group Members:

        asg_cp2blades /home/admin/payload.json

      3. Run this command:

        • On a Security Gateway / each Cluster Member / Management Server / Log Server / SmartEvent Server:

          sklnctl export --set "$(cat /home/admin/payload.json)"

        • On a Scalable Platform Security Group:

          g_all sklnctl export --set "$(cat /home/admin/payload.json)"

    • Method 2 - Run the Gaia REST API command "set-open-telemetry" (requires Gaia API v1.7 and higher):

      POST{server}/v1.7/set-open-telemetry
Content-Type: application/json
X-chkp-sid: {{session}}
{
   <JSON Payload>
}

      Note - To disable Skyline completely:

      1. Change the value of the "enabled" attribute in the JSON payload file:

        from "enabled": true

        to "enabled": false

      2. Run the API command again

  1. Prepare the required payload for the command:

    Notes:

    • Refer to the AWS official documentation for detailed information about how to generate the keys.

    • The Session Token is optional.

    To work with a single Export Target over HTTPS:

    {
    "enabled": true,
    "export-targets": {
        "add": [
            {
                "server-auth": {
                    "sigv4auth": {
                        "region": "<Region>",
                        "aws-access-key-id": "<Access Key ID>",
                        "aws-secret-access-key": "<Access Key>",
                        "session-token": "<Session Token>"
                    }
                },
                "enabled": true,
                "type": "prometheus-remote-write",
                "url": "https://<PROMETHEUS_IP_ADDRESS>:9090/api/v1/write"
            }
        ]
    }
}

    To work with multiple Export Targets over HTTPS (this example shows two):

    {
    "enabled": true,
    "export-targets": {
        "add": [
            {
                "server-auth": {
                    "sigv4auth": {
                        "region": "<Region>",
                        "aws-access-key-id": "<Access Key ID>",
                        "aws-secret-access-key": "<Access Key>",
                        "session-token": "<Session Token>"
                    }
                },
                "enabled": true,
                "type": "prometheus-remote-write-1",
                "url": "https://<PROMETHEUS_IP_ADDRESS_#1>:9090/api/v1/write",
                "name": "My-Prometheus-Target-1"
            },
            {
                "enabled": true,
                "type": "prometheus-remote-write-2",
                "url": "https://<PROMETHEUS_IP_ADDRESS_#2>:9090/api/v1/write",
                "name": "My-Prometheus-Target-2"
            }
        ]
    }
}

    To work with multiple Export Targets over HTTPS and over HTTP (this example shows two):

    {
    "enabled": true,
    "export-targets": {
        "add": [
            {
                "server-auth": {
                    "sigv4auth": {
                        "region": "<Region>",
                        "aws-access-key-id": "<Access Key ID>",
                        "aws-secret-access-key": "<Access Key>",
                        "session-token": "<Session Token>",
                        "enabled": true,
                        "type": "prometheus-remote-write-1",
                        "url": "https://<PROMETHEUS_IP_ADDRESS_#1>:9090/api/v1/write",
                        "name": "My-Prometheus-Target-1"
                    }
                }
            },
            {
                "enabled": true,
                "type": "prometheus-remote-write-2",
                "url": "http://<PROMETHEUS_IP_ADDRESS_#2>:9090/api/v1/write",
                "name": "My-Prometheus-Target-2"
            }
        ]
    }
}

  2. Run the configuration command to apply the payload - either the CLI command (in the Expert mode), or the Gaia REST API command:

    • Method 1 - Run the CLI command "sklnctl":

      1. Save the JSON payload in a file (for example, /home/admin/payload_AWS.json).

      2. On a Scalable Platform Security Group, copy the JSON payload file to all Security Group Members:

        asg_cp2blades /home/admin/payload_AWS.json

      3. Run this command:

        • On a Security Gateway / each Cluster Member / Management Server / Log Server / SmartEvent Server:

          sklnctl export --set "$(cat /home/admin/payload_AWS.json)"

        • On a Scalable Platform Security Group:

          g_all sklnctl export --set "$(cat /home/admin/payload_AWS.json)"

    • Method 2 - Run the Gaia REST API command "set-open-telemetry" (requires Gaia API v1.7 and higher):

      POST{server}/v1.7/set-open-telemetry
Content-Type: application/json
X-chkp-sid: {{session}}
{
   <JSON AWS Payload>
}

Step 5 - Configure the filter for the OpenTelemetry Collector exported metrics

The sklnctl tool configures the OpenTelemetry Collector.

The OpenTelemetry Collector filter works on the allow-list basis.

Note - On a Scalable Platform Security Group, run the "g_all sklnctl <options>" command.

These are the available commands (in the Expert mode):

  • To show the currently exported metrics:

    sklnctl otelcol metrics --show

  • To check if default metrics are exported:

    sklnctl otelcol metrics --is-default

  • To add metrics to the allow-list:

    sklnctl otelcol metrics --add <metric-id1> <metric-id2> <metric-id3> ...

  • To remove metrics from the allow-list:

    sklnctl otelcol metrics --remove <metric-id1> <metric-id2> <metric-id3> ...

  • To reset the allow-list to the default:

    sklnctl otelcol metrics --reset

Example workflow (in the Expert mode):

  1. Show the currently exported metrics:

    sklnctl otelcol metrics --show > /var/log/metrics.txt

  2. Edit the file to keep only the desired metrics:

    vi /var/log/metrics.txt

    See Skyline Metrics Repository.

  3. On a Scalable Platform Security Group, copy the file to all Security Group Members:

    asg_cp2blades /var/log/metrics.txt

  4. Add the desired metrics to the allow-list:

    • On a Security Gateway / each Cluster Member / Management Server / Log Server / SmartEvent Server:

      sklnctl otelcol metrics --add $(cat /var/log/metrics.txt | tr '\n' ' ')

    • On a Scalable Platform Security Group:

      g_all sklnctl otelcol metrics --add $(cat /var/log/metrics.txt | tr '\n' ' ')

Step 6 - Configure Access Control Policy

If you configured Skyline on a Security Gateway, ClusterXL, or Scalable Platform Security Group, then you must make sure your Access Control Policy allows the connection to the Prometheus Server to send the exported metrics.

You must configure the required rule on the Management Server (in SmartConsole or with Management API) and install the policy.

See the: