Grafana Alerts

Overview of Alerts

Grafana alerts allow you to turn the observability dashboards into a proactive monitoring system.

Instead of relying only on manual dashboard checks, you can configure rules that automatically notify you when SD-WAN metrics indicate a problem.

The Alerts are prioritized based on importance. Make sure the label is configured on all the alert rules.

For example:

  • No telemetry received from a Security Gateway

  • A WAN link went down or changes its status frequently (flapping)

  • Link QoE or MOS score decreases below a threshold

  • VPN tunnels going down while the underlay link is still up

  • Characteristics of steering objects are exceeding the thresholds for latency / jitter / packet loss

  • High CPU or memory utilization on Security Gateways

With these alerts in place, NOC and operations teams can detect issues early, prioritize troubleshooting, and often resolve problems before end users start to complain.

These Grafana-based alerts are complementary to Check Point Infinity alerts:

  • Infinity alerts Includes the Grafana Alerts as part of AI-Ops Alerts but also focus mainly on security events, management-level status and product-level health (threats, policy, blades, etc.) across the Check Point environment.

  • Grafana SD-WAN alerts are built directly on top of the Skyline/telemetry metrics used by the dashboards and are tailored to network and SD-WAN performance - underlay link quality, overlay tunnel health, QoE/MOS scores, and SD-WAN steering behavior.

Because they are defined in Grafana, these alerts are:

  • Fully customizable - you can change thresholds, add new rules or disable the ones you do not need.

  • Independent of Infinity - useful for customers who want local, high-frequency operation alerts alongside central Infinity alerts.

  • Directly linked to the dashboards - an alert can be opened from Grafana and immediately correlated with the corresponding panels with Security Gateway, ISP links, or VPN tunnels.

Import Alerts into Grafana

  1. Download the required YAML file from sk180605 > section "SD-WAN Observability.

  2. In Grafana Go to Alerting -> Alert Rules, on the upper right side select more – Import alert rules

  3. Under import source select Prometheus YAML file and upload the file, select the target data source. In additional settings select a folder or create one. Uncheck the Pause Import Alert Rules

    Under Recording rules select the data source to save alert statuses and import.

  4. On the window "Confirm import" click Import:

  5. 5. If you didn’t uncheck the pause all rules on stage 3.On the right side select actions resume all rules:

Writing Grafana Alerts to Prometheus

  1. To enable the alert counter on the Underlay tab, edit the Grafana configuration file to write the alert status into Prometheus.

    Usually, the configuration file is: /etc/grafana/grafana.ini

    If it is a docker container, then use this command:

    docker exec -it -u root grafana vi /etc/grafana/grafana.ini

  2. Find this section:

    [unified_alerting.state_history]

  3. In this section, make sure these lines exist (add the required lines) and they are uncommented (must not start with the ";" character):

    enabled = true

    backend = prometheus

    prometheus_target_datasource_uid = <UID of Prometheus>

    Example:

  4. Save the changes in the file and exit the editor.

  5. Restart the container / Grafana service.

  6. Restart the Docker container:

    docker restart <Name of Grafana Docker Container>

    Example:

    docker restart grafana

Downloading the SD-WAN Alert File

Download the required YAML file from sk180605 > section "SD-WAN Observability.

In the YAML file, change the Prometheus UID to your Prometheus UID.

You can see the UID of your Prometheus in the URL when you are in the Prometheus data source configuration page.

Example:

Next Step:

See Manual Configuration of Alerts.