Zyhel

NikitaHarashchuk
NikitaHarashchuk Posts: 2  Freshman Member
in Security

Hello, I need help configuring L2TP/IPSec VPN on Zyxel USG FLEX 100.

Problem description:

I configured L2TP VPN via the Quick Setup Wizard for remote access.
Using Windows 11 client, authentication method PSK.
When connecting, Windows shows:

"The L2TP connection attempt failed because of a security layer error occurred during initial negotiations with the remote computer."

In the Zyxel logs I can see the connection drops right after NONCE exchange in IKE phase (SA 1 never comes up).

Log fragment:

IKE SA [] is disconnected
185.152.139.170:500
23.139.82.62:47084
...
[INIT] Recv: [NOTIFY][NONCE]
Receiving IKEv2 request

What I have tried:

  • Configured VPN via Wizard.
  • Tested connection from LAN and external network — same result.
  • Double-checked PSK, IKE and IPSec services are enabled.
  • Allowed/forwarded UDP 500/4500/1701 on WAN interface.
  • Followed all steps from winitpro.ru guide.

The strange part:

In Configuration > VPN > IPSec VPN > VPN Gateway, under Encryption, the only available option is DES.

image.png

AES128/256 and 3DES are completely missing.

The issue is that:

  • Windows client by default uses DES/SHA1/DH2, so DES should match, but the tunnel never comes up.
  • On older firmware versions (based on community feedback) AES/3DES could be selected.
  • On my firmware (V5.40(ABUH.0), 2025-05-07) only DES is available.
image.png

Questions:

  1. Is this a bug in current firmware V5.40, or is it intentional (DES-only left for compatibility)?
  2. Is there a way to add AES/3DES to IPSec Proposal for L2TP via CLI?
  3. Should I downgrade to V5.39 (or earlier) where AES/3DES was available?
  4. Am I maybe looking in the wrong place, and encryption proposals are configured separately?

Configuration details:

  • Model: USG FLEX 100
  • Firmware: V5.40(ABUH.0) (2025-05-07)
  • Standby: V5.39(ABUH.1) (2024-11-16)
  • Client: Windows 11 (L2TP/IPSec PSK)

All Replies

  • PeterUK
    PeterUK Posts: 4,020  Guru Member
    100 Answers 2500 Comments Friend Collector Seventh Anniversary

    Not sure why you only see DES

    Delete the Wizard setup and do it manually

    windows default is
    Phase 1
    3DES SHA1
    key group DH2
    Phase 2
    AES256 SHA1
    PFS none

    Phase 1 VPN gateway
    negotiation mode = main

    Phase 2 VPN connection
    Remote Access (Server Role)
    local policy 0.0.0.0
    encapsulation Transport

    setup L2TP over IPSec

  • Zyxel_Tina
    Zyxel_Tina Posts: 202  Zyxel Employee
    Zyxel Certified Network Administrator - Security Zyxel Certified Network Administrator - Switch 50 Answers First Comment
    edited August 29

    Hi @NikitaHarashchuk,

    We’ve tested the same model (USG FLEX 100) with the same firmware version V5.40(ABUH.0), and after configuring the VPN via the Wizard, going to Configuration > VPN > IPSec VPN > VPN Gateway, we can see the Phase 1 proposal is 3DES/SH1 and the full set of encryption options (DES/3DES/AES128/AES192/AES256).

    image.png

    To move forward, we recommend:

    1. Backup your configuration first.
    2. Double-check whether your current VPN settings fully match the wizard-created config. image.png
    3. If possible, try deleting the existing VPN configuration and recreating it from scratch using the Wizard.
    4. If the issue still persists, please refer to the commands shown in the configuration image above and set them up via CLI.

    Zyxel Tina

Categories

Security Help Center

EnglishTiếng ViệtРусскийไทย中文（台灣）