IKEv2 fragmentation support in ATP firewalls

dbastas
dbastas Posts: 7  Freshman Member
First Comment Friend Collector Fifth Anniversary

Hi everyone,

I have a weird problem setting up an IKEv2 VPN on a ATP firewall using a self signed certificate from the same ATP.
Some users from some places can connect to the IKEv2 VPN and some others from other places don't.
All client are using the same Windows build.
I think the problem have to do with the IKEv2 fragmentation on IKE Phase 1.
I captured packets during the creation of the tunnel and saw that the Windows client sent this payload on phase 1
Payload: Notify (41) - IKEV2_FRAGMENTATION_SUPPORTED
but the reply from the ATP did not send this.
So I would like to ask if the ATP firewall supports IKEv2 fragmentation during phase 1 and if so how to enable this functionality.

«1

All Replies

  • Zyxel_Tina
    Zyxel_Tina Posts: 155  Zyxel Employee
    Zyxel Certified Network Administrator - Security Zyxel Certified Network Administrator - Switch 25 Answers First Comment

    Hi @dbastas,

    Welcome to Zyxel Community!

    Unfortunately, Zyxel ATP firewalls do not support IKEv2 fragmentation.

    To help us further investigate and better assist you, we kindly ask:

    • Could you share a screenshot where you observed the Notify (41) - IKEV2_FRAGMENTATION_SUPPORTED payload from the Windows client, and the corresponding response from the ATP?

    With this additional information, we’ll be in a better position to determine whether the problem is indeed related to IKEv2 fragmentation or if other factors are involved.

    We appreciate your cooperation!

    Zyxel Tina

  • PeterUK
    PeterUK Posts: 3,963  Guru Member
    100 Answers 2500 Comments Friend Collector Seventh Anniversary

    I'm not sure why you say you do not support IKEv2 fragmentation?

    Here is a IKEv2 by Certificate between between VPN300 and FLEX200

    Screenshot 2025-07-31 121747.png

    Now on the FLEX200 H I think fragmentation is done by UDP with ports but older Zyxel models do fragmentation without ports which has a problem should between end points fragmentation by no ports are blocked.

  • dbastas
    dbastas Posts: 7  Freshman Member
    First Comment Friend Collector Fifth Anniversary

    Hi all,

    To clear any misunderstandings, when I say IKEv2 fragmentation I am talking about RFC 7383 (https://datatracker.ietf.org/doc/html/rfc7383)
    This screenshot (frame 5) is the IKE_SA_INIT packet from Windows client towards the ATP firewall

    image.png

    This screenshot (frame 6) is firewall's reply to IKE_SA_INIT packet from ATP firewall towards the Windows client

    image.png

    We clearly observe the lack of payload IKEV2_FRAGMENTATION_SUPPORTED from firewall reply

  • Zyxel_Tina
    Zyxel_Tina Posts: 155  Zyxel Employee
    Zyxel Certified Network Administrator - Security Zyxel Certified Network Administrator - Switch 25 Answers First Comment
    edited August 6

    Hi @dbastas and @PeterUK,

    After double checking, our firewall does not support IKEv2 fragmentation. However, this lack of support does not affect the overall VPN functionality. Therefore, it is expected that you do not see the IKEv2_fragmentation_supported notification payload in the packets from the ATP towards the Windows client.

    Regarding the Fragmented IP protocol packets captured by @PeterUK, these do not indicate support for fragmentation. For a true indication of IKEv2 fragmentation support, please refer to the information on page 11 of RFC 7383.

    Zyxel Tina

  • dbastas
    dbastas Posts: 7  Freshman Member
    First Comment Friend Collector Fifth Anniversary

    Hi all,

    In fact there is a problem if the VPN server (in this case the ATP firewall) does not support RFC 7383.
    In that case some times the windows client connects to the VPN server and some times does not, with the error message 'Error code 809'
    During IKEv2 connection establishment, payload sizes may exceed the IP Maximum Transmission Unit (MTU) for the network path between the client and server. This causes the IP packets to be fragmented. However, it is not uncommon for intermediary devices (routers, NAT devices, or firewalls) to block IP fragments. When this occurs, a VPN connection cannot be established.
    IKEv2 fragmentation was introduced in Windows 10 1803 and is enabled by default. No client-side configuration is required.
    In my case the same laptop with the exact same configuration connects to the IKEv2 VPN from one location but not from another location.

  • Zyxel_Tina
    Zyxel_Tina Posts: 155  Zyxel Employee
    Zyxel Certified Network Administrator - Security Zyxel Certified Network Administrator - Switch 25 Answers First Comment
    edited August 13

    Hi @dbastas,

    Sorry for the late reply.

    Regarding the error code 809 issue, we would like to clarify the possible issue first. Could you please check the following to help us further investigate:

    1. From the Windows client’s uplink network, please verify that VPN traffic is allowed to pass through. Based on your earlier screenshot, UDP port 500 for IKEv2 appears to be allowed, so please test UDP port 4500 using the tool at https://portquiz.net:4500 to ensure it’s not being blocked.
    2. Kindly provide the ATP firewall event log around the time of the failed connection attempts.
    3. Please also share your ATP model name and firmware version.

    Thank you for your cooperation!

    Zyxel Tina

  • dbastas
    dbastas Posts: 7  Freshman Member
    First Comment Friend Collector Fifth Anniversary

    Hi @ZyXEL _Tina

    Rest assured that the port 4500/udp is open on the ATP firewall. The L2TP VPN is working without a problem on the same firewall using the same laptop. The reason I want to setup an IKEv2 VPN is that there are mobile VPN clients that need to connect using a VPN and the newer Android versions stopped supporting L2TP VPN. This is is the result of portquiz.net for this port:

    image.png

    These are the ATP firewall relevant log at the moment the client tries to connect using IKEv2 VPN:

    image.png

    My firewall is an ATP500 and the firmware version is V5.39(ABFU.1)

  • PeterUK
    PeterUK Posts: 3,963  Guru Member
    100 Answers 2500 Comments Friend Collector Seventh Anniversary
    edited August 19

    Android built in client may not work well for Certificates last time I checked so try built in client Pre-Shared Key (IKEv2/IPSec PSK) without Enable Extended Authentication Protocol so no need for user or password.

    If you have a DDNS for clients to connect too be sure to Local ID type DNS one ATP and on client for settings IPSec identifier as ikev2

    Then you need to setup Phase 1 and 2 Proposal to which the following work for me:
    Phase 1
    AES128 SHA256
    DH14
    Phase 2
    AES128 SHA256
    DH2

    if you need a VPN client with Certificates and user name support use strongSwan

  • dbastas
    dbastas Posts: 7  Freshman Member
    First Comment Friend Collector Fifth Anniversary

    Thanks for the tip.
    All the trials were done using a laptop running windows 10 using the built in VPN client. When there will be a final solution for this problem we intend to use strongSwan on Android devices. The problem with the IKEv2 with Pre-Shared keys is that Microsoft supports only IKEv2 with certificates on Windows machines.
    I suspect that IKEv2 with Pre-Shared keys works because of the much smaller payload in the negotiation phase, so there is no need for IKEv2 fragmentation.

  • PeterUK
    PeterUK Posts: 3,963  Guru Member
    100 Answers 2500 Comments Friend Collector Seventh Anniversary
    edited August 19

    But windows default Phase 1 and 2 is different too strongSwan so you need to config windows by Windows PowerShell with this

    Set-VpnConnectionIPsecConfiguration -ConnectionName "connection name" -AuthenticationTransformConstants SHA256128 -CipherTransformConstants AES256 -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -PfsGroup PFS2 -DHGroup Group14 -PassThru -Force

    Set-VpnConnectionIPsecConfiguration (VpnClient) | Microsoft Learn

    Phase 1
    AES256 SHA256
    DH14
    Phase 2
    AES256 SHA256
    DH2

    are you use a self sign cert or real? and IP or DNS?