Contents/Index/Search Download Complete PDF Send Feedback Print This Page




Related Topics


IKE DoS Protection

Configuring Advanced IKE Properties


In symmetric cryptographic systems, both communicating parties use the same key for encryption and decryption. The material used to build these keys must be exchanged in a secure fashion. Information can be securely exchanged only if the key belongs exclusively to the communicating parties.

The goal of the Internet Key Exchange (IKE) is for both sides to independently produce the same symmetrical key. This key then encrypts and decrypts the regular IP packets used in the bulk transfer of data between VPN peers. IKE builds the VPN tunnel by authenticating both sides and reaching an agreement on methods of encryption and integrity. The outcome of an IKE negotiation is a Security Association (SA).

This agreement upon keys and methods of encryption must also be performed securely. For this reason, IKE is composed of two phases. The first phase lays the foundations for the second. Both IKEv1 and IKEv2 are supported in Security Gateways of version R71 and higher.

Diffie-Hellman (DH) is that part of the IKE protocol used for exchanging the material from which the symmetrical keys are built. The Diffie-Hellman algorithm builds an encryption key known as a "shared secret" from the private key of one party and the public key of the other. Since the IPSec symmetrical keys are derived from this DH key shared between the peers, at no point are symmetric keys actually exchanged.

IKE Phase I

During IKE Phase I:

  • The peers authenticate, either by certificates or via a pre-shared secret. (More authentication methods are available when one of the peers is a remote access client.)
  • A Diffie-Hellman key is created. The nature of the Diffie-Hellman protocol means that both sides can independently create the shared secret, a key which is known only to the peers.
  • Key material (random bits and other mathematical data) as well as an agreement on methods for IKE phase II are exchanged between the peers.

In terms of performance, the generation of the Diffie Hellman Key is slow and heavy. The outcome of this phase is the IKE SA, an agreement on keys and methods for IKE phase II. Figure 2‑1 illustrates the process that takes place during IKE phase I but does not necessarily reflect the actual order of events.

IKE Phase II (Quick mode or IPSec Phase)

IKE phase II is encrypted according to the keys and methods agreed upon in IKE phase I. The key material exchanged during IKE phase II is used for building the IPSec keys. The outcome of phase II is the IPSec Security Association. The IPSec SA is an agreement on keys and methods for IPSec, thus IPSec takes place according to the keys and methods agreed upon in IKE phase II.

After the IPSec keys are created, bulk data transfer takes place:

IKEv1 and IKEv2

IKEv2 is supported inside VPN communities working in Simplified mode in versions R71 and higher.

IKEv2 is configured in the VPN Community Properties window > Encryption. The default setting is IKEv1 only. IKEv2 is automatically always used for IPv6 traffic. The encryption method configuration applies to IPv4 traffic only.

For Remote users, the IKE settings are configured in Global Properties > Remote Access > VPN Authentication and Encryption.

Note - IKEv2 is not supported on UTM-1 Edge devices or VSX objects before R75.40VS. If UTM-1 Edge devices or such VSX objects are included in a VPN Community, the Encryption setting should be Support IKEv1.

Methods of Encryption and Integrity

Two parameters are decided during the negotiation:

  • Encryption algorithm
  • Hash algorithm


IKE Phase 1 (IKE SA)



  • AES-128
  • AES-256(default)
  • 3DES
  • DES
  • CAST (IKEv1 only)
  • AES-128 (default)
  • AES-256
  • 3DES
  • DES
  • DES-40CP (IKEv1 only)
  • CAST (IKEv1 only)
  • CAST-40 (IKEv1 only)
  • NULL
  • AES-GCM-128
  • AES-GCM-256


  • MD5
  • SHA1 (default)
  • SHA -256
  • SHA -384
  • MD5
  • SHA1 (default)
  • SHA -256
  • SHA -384

NULL means perform an integrity check only; packets are not encrypted.

Diffie Hellman Groups

The Diffie-Hellman key computation (also known as exponential key agreement) is based on the Diffie Hellman (DH) mathematical groups. A Security Gateway supports these DH groups during the two phases of IKE.


IKE Phase 1 (IKE SA)

IKE Phase 2 (IPSec SA)

Diffie Hellman Groups

  • Group2 (1024 bits) (default)
  • Group1 (768 bits)
  • Group5 (1536 bits)
  • Group14 (2048 bits)
  • Group19 (256-bit ECP)
  • Group20 (384-bit ECP)
  • Group2 (1024 bits) (default)
  • Group1 (768 bits)
  • Group5 (1536 bits)
  • Group14 (2048 bits)
  • Group19 (256-bit ECP)
  • Group20 (384-bit ECP)

A group with more bits ensures a key that is harder to break, but carries a heavy cost in terms of performance, since the computation requires more CPU cycles.

Phase I modes

Between Security Gateways, there are two modes for IKE phase I. These modes only apply to IKEv1:

  • Main Mode
  • Aggressive Mode

If aggressive mode is not selected, the Security Gateway defaults to main mode, performing the IKE negotiation using six packets; aggressive mode performs the IKE negotiation with three packets.

Main mode is preferred because:

  • Main mode is partially encrypted, from the point at which the shared DH key is known to both peers.
  • Main mode is less susceptible to Denial of Service (DoS) attacks. In main mode, the DH computation is performed after authentication. In aggressive mode, the DH computation is performed parallel to authentication. A peer that is not yet authenticated can force processor intensive Diffie-Hellman computations on the other peer.

Note - Use aggressive mode when a Check Point Security Gateway needs to negotiate with third party VPN solutions that do not support main mode.

When dealing with remote access, IKE has additional modes:

  • Hybrid mode. Hybrid mode provides an alternative to IKE phase I, where the Security Gateway is allowed to authenticate using certificates and the client via some other means, such as SecurID. For more information on Hybrid mode, see: Introduction to Remote Access VPN.
  • Office mode. Office mode is an extension to the IKE protocol. Office Mode is used to resolve routing issues between remote access clients and the VPN domain. During the IKE negotiation, a special mode called config mode is inserted between phases I and II. During config mode, the remote access client requests an IP address from the Security Gateway. After the Security Gateway assigns the IP address, the client creates a virtual adapter in the Operating System. The virtual adapter uses the assigned IP address. For further information, see: Office Mode.

Renegotiating IKE & IPSec Lifetimes

IKE phase I is more processor intensive than IKE phase II, since the Diffie-Hellman keys have to be produced and the peers authenticated each time. For this reason, IKE phase I is performed less frequently. However, the IKE SA is only valid for a certain period, after which the IKE SA must be renegotiated. The IPSec SA is valid for an even shorter period, meaning many IKE phase II's take place.

The period between each renegotiation is known as the lifetime. Generally, the shorter the lifetime, the more secure the IPSec tunnel (at the cost of more processor intensive IKE negotiations). With longer lifetimes, future VPN connections can be set up more quickly. By default, IKE phase I occurs once a day; IKE phase II occurs every hour but the time-out for each phase is configurable.

The IPSec lifetime can also be configured according to Kilo Bytes by using GuiDBedit to edit the objects_5_0.c file. The relevant properties are under the community set:

  • ike_p2_use_rekey_kbytes. Change from false (default) to true.
  • ike_p2_rekey_kbytes. Modify to include the required rekeying value (default 50000).

Perfect Forward Secrecy

The keys created by peers during IKE phase II and used for IPSec are based on a sequence of random binary digits exchanged between peers, and on the DH key computed during IKE phase I.

The DH key is computed once, then used a number of times during IKE phase II. Since the keys used during IKE phase II are based on the DH key computed during IKE phase I, there exists a mathematical relationship between them. For this reason, the use of a single DH key may weaken the strength of subsequent keys. If one key is compromised, subsequent keys can be compromised with less effort.

In cryptography, Perfect Forward Secrecy (PFS) refers to the condition in which the compromise of a current session key or long-term private key does not cause the compromise of earlier or subsequent keys. Security Gateways meet this requirement with a PFS mode. When PFS is enabled, a fresh DH key is generated during IKE phase II, and renewed for each key exchange.

However, because a new DH key is generated during each IKE phase I, no dependency exists between these keys and those produced in subsequent IKE Phase I negotiations. Enable PFS in IKE phase II only in situations where extreme security is required.

The DH group used during PFS mode is configurable between groups 1, 2, 5 and 14, with group 2 (1042 bits) being the default.

Note - PFS mode is supported only between gateways, not between Security Gateways and remote access clients.

IP Compression

IP compression is a process that reduces the size of the data portion of the TCP/IP packet. Such a reduction can cause significant improvement in performance. IPSec supports the Flate/Deflate IP compression algorithm. Deflate is a smart algorithm that adapts the way it compresses data to the actual data itself. Whether to use IP compression is decided during IKE phase II. IP compression is not enabled by default.

IP compression is important for SecuRemote/SecureClient users with slow links. For Example, dialup modems do compression as a way of speeding up the link. Security Gateway encryption makes TCP/IP packets appear "mixed up". This kind of data cannot be compressed and bandwidth is lost as a result. If IP compression is enabled, packets are compressed before encryption. This has the effect of recovering the lost bandwidth.

Subnets and Security Associations

By default, a VPN tunnel is created for the complete subnets that host computers reside on, and not just for the host computers involved in the communication.

A Security Gateway protects a network consisting of two subnets (10.10.10.x, and 10.10.11.x, with netmask for both). A second Security Gateway, the remote peer, protects subnets 10.10.12.x and 10.10.13.x, with netmask

Because a VPN tunnel is created by default for complete subnets, four SA's exist between the Security Gateway and the peer Security Gateway. When Host A communicates with Host B, an SA is created between Host A's subnet and Host B's subnet.

Unique SA Per Pair of Peers

By disabling the Support Key exchange for subnets option on each Security Gateway, it is possible to create a unique Security Association per pair of peers.

If the Security Gateway is configured to Support key exchange for subnets and the option remains unsupported on the remote peer, when host A communicates with host C, a Security Association (SA 1) will be negotiated between host A's subnet and host C's IP address. The same SA is then used between any host on the 10.10.11.x subnet and Host C.

When host A communicates with host B, a separate Security Association (SA 2) is negotiated between host A's subnet and host B. As before, the same SA is then used between any host in 10.10.11.x subnet and Host B.

When Support Key exchange for subnets is not enabled on communicating Security Gateways, then a security association is negotiated between individual IP addresses; in effect, a unique SA per host.

IKE DoS Protection

Understanding DoS Attacks

Denial of Service (DoS) attacks are intended to reduce performance, block legitimate users from using a service, or even bring down a service. They are not direct security threats in the sense that no confidential data is exposed, and no user gains unauthorized privileges. However, they consume computer resources such as memory or CPU.

Generally, there are two kinds of DoS attack. One kind consists of sending malformed (garbage) packets in the hope of exploiting a bug and causing the service to fail. In the other kind of DoS attack, an attacker attempts to exploit a vulnerability of the service or protocol by sending well-formed packets. IKE DoS attack protection deals with the second kind of attack.

IKE DoS Attacks

The IKE protocol requires that the receiving Security Gateway allocates memory for the first IKE Phase 1 request packet that it receives. The Security Gateway replies, and receives another packet, which it then processes using the information gathered from the first packet.

An attacker can send many IKE first packets, while forging a different source IP address for each. The receiving Security Gateway is obliged to reply to each, and assign memory for each. This can consume all CPU resources, thereby preventing connections from legitimate users.

The attacker sending IKE packets can pretend to be a machine that is allowed to initiate IKE negotiations, such as a Check Point Security Gateway. This is known as an identified source. The attacker can also pretend to have an IP address that the receiving Security Gateway does not know about, such as a SecuRemote/SecureClient, or a Check Point Security Gateway with a dynamic IP address. This is known as an unidentified source.

Defense Against IKE DoS Attacks

When the number of simultaneous IKE negotiations handled exceeds the accepted threshold, it concludes that it is either under load or experiencing a Denial of Service attack. In such a case, the Security Gateway can filter out peers that are the probable source of a potential Denial of Service attack. The following sections describe different types of defenses against IKE DoS attacks.

IKE DoS protection is not supported for IPv6 addresses.

SmartDashboard IKE DoS Attack Protection Settings

To protect against IKE DoS attacks, configure the SmartDashboard IKE Denial of Service Protection settings, in the VPN >Advanced page of the Global Properties. IKE DoS protection is not supported for IPv6 addresses.

  • Support IKE DoS protection from identified source — The default setting for identified sources is Stateless. If the Security Gateway is under load, this setting requires the peer to respond to an IKE notification in a way that proves that the IP address of the peer is not spoofed. If the peer cannot prove this, the Security Gateway does not begin the IKE negotiation.

    If the source is identified, protecting using Puzzles is over cautious, and may affect performance. A third possible setting is None, which means no DoS protection.

  • Support IKE DoS protection from unidentified source — The default setting for unidentified sources is Puzzles. If the Security Gateway is under load, this setting requires the peer to solve a mathematical puzzle. Solving this puzzle consumes peer CPU resources in a way that makes it difficult to initiate multiple IKE negotiations simultaneously.

    For unidentified sources, Stateless protection may not be sufficient because an attacker may well control all the IP addresses from which the IKE requests appear to be sent. A third possible setting is None, which means no DoS protection.

Advanced IKE DoS Attack Protection Settings

Advanced IKE DoS attack protection can be configured on the Security Management server using the dbedit command line or using GuiDBedit, the Check Point Database Tool. Configure the protection by means of the following Global Properties. IKE DoS protection is not supported for IPv6.


Values: 0-100. Default: 70. Determines the percentage of maximum concurrent ongoing negotiations, above which the Security Gateway will request DoS protection. If the threshold is set to 0, the Security Gateway will always request DoS protection.


Values: 0-32. Default: 19. Determines the level of the puzzles sent to known peer Security Gateways. This attribute also determines the maximum puzzle level a Security Gateway is willing to solve.


Values: 0-32. Default: 19. Determines the level of the puzzles sent to unknown peers (such as SecuRemote/SecureClients and DAIP Security Gateways). This attribute also determines the maximum puzzle level that DAIP Security Gateways and SecuRemote/SecureClients are willing to solve.


Values: 0-30000. Default: 500. Determines the maximum time in milliseconds a Security Gateway is willing to spend solving a DoS protection puzzle.


Values: 0-30000. Default: 500. Determines the maximum time in milliseconds a DAIP Security Gateway is willing to spend solving a DoS protection puzzle.


Values: 0-30000. Default: 5000. Determines the maximum time in milliseconds a SecuRemote is willing to spend solving a DoS protection puzzle.


Values: None, Stateless, Puzzles. Default: Puzzles. When downloaded to SecuRemote/SecureClient, it controls the level of protection the client is willing to support.

Security Gateways use the ike_dos_protection_unidentified_initiator property (equivalent to the SmartDashboard Global Property: Support IKE DoS Protection from unidentified Source) to decide what protection to require from remote clients, but SecuRemote/SecureClient clients use the ike_dos_protection. This same client property is called ike_dos_supported_protection_sr on the Security Gateway.

Protection After Successful Authentication

You can configure fields in GuiDBedit to protect against IKE DoS attacks from peers who may authenticate successfully and then attack a Security Gateway. These settings are configured in the Global Properties table and not per Security Gateway. By default these protections are off. Once you enter a value, they will be activated.

To limit the amount of IKE Security Associations (SA's) that a user can open, configure the following fields:

Type of VPN


Recommended Value

Site to site



Remote user



To limit the amount of tunnels that a user can open per IKE, configure the following fields:

Type of VPN


Recommended Value

Site to site



Remote user



Client Properties

Some Security Gateway properties change name when they are downloaded to SecuRemote/SecureClient. The modified name appears in the Userc.C file, as follows:

Property Names

Property Name on Gateway

User.C Property name on Client

(Equivalent to the SmartDashboard Global Property: Support IKE DoS Protection from unidentified Source)

ike_dos_protection or







Configuring Advanced IKE Properties

IKE is configured in two places:

  • On the VPN community network object (for IKE properties).
  • On the Security Gateway network object (for subnet key exchange).

On the VPN Community Network Object

IPv6 automatically works with IKE v2 encryption only. The option that you select here, applies to IPv4 traffic.

  1. From the VPN Community Properties > Encryption page, select:
    • Encryption Method - For IKE phase I and II.
      • IKEv2 only - Only support encryption using IKEv2. Security Gateways in this community cannot access peer gateways that support IKEv1 only.
      • Prefer IKEv2, support IKEv1 - If a peer supports IKEv2, the Security Gateway will use IKEv2. If not, it will use IKEv1 encryption. This is recommended if you have a community of older and new Check Point Security Gateways.
      • IKEv1 only - IKEv2 is not supported.
    • Encryption Suite - The methods negotiated in IKE phase 2 and used in IPSec connections. Select the option for best interoperability with other vendors in your environment.
      • VPN-A or VPN B - See RFC 4308 for more information.
      • Suite-B GCM-128 or 256 - See RFC 6379 for more information.
      • If you require algorithms other than those specified in the other options, select Custom and click Advanced to select properties for IKE Phase 1 and 2.

      Note - Suite-B GCM-128 and 256 encryption suites are supported on R71.50 gateways and from R75.40 gateways.

  2. From the VPN Community Properties > Advanced Settings > Advanced VPN Properties page, select:
    • Which Diffie-Hellman group to use.
    • When to renegotiate the IKE Security Associations.
    • Whether to use aggressive mode (Main mode is the default).
    • Whether to use Perfect Forward Secrecy, and with which Diffie-Hellman group.
    • When to renegotiate the IPSec security associations.
    • Whether to use Support IP compression.
    • Whether to disable NAT inside the VPN community.

On the Gateway Network Object

  1. On the IPSec VPN > VPN Advanced page, select one of the options in the VPN Tunnel Sharing section. There are several settings that control the number of VPN tunnels between peer gateways:

    Note - Wire Mode is not supported for IPv6 connections.

    • Use the community settings - Create the number of VPN tunnels as defined on the community Tunnel Management page.
    • Custom settings:
      • One VPN tunnel per each pair of hosts - A VPN tunnel is created for every session initiated between every pair of hosts.
      • One VPN tunnel per subnet pair - Once a VPN tunnel has been opened between two subnets, subsequent sessions between the same subnets will share the same VPN tunnel. This is the default setting and is compliant with the IPSec industry standard.
      • One VPN tunnel per Gateway pair - One VPN tunnel is created between peer gateways and shared by all hosts behind each peer gateway.
  2. On the Capacity Optimization page, you can maximize VPN throughput by limiting the following connection parameters:
    • Maximum concurrent IKE negotiations
    • Maximum concurrent runnels

    If you have many employees working remotely, you may want to raise the default values.

Top of Page ©2013 Check Point Software Technologies Ltd. All rights reserved. Download Complete PDF Send Feedback Print