How to authenticate for any VPN Connection by certificates with the built-in MacOS VPN-Client?

tomaltmann Posts: 2
Friend Collector First Comment
edited April 2021 in Security

Hey Guys,

I'm using a Zyxel ZyWall 110 and I want to establish a client-to-side VPN connection to the ZyWall by using the built-in VPN-Client from MacOS 10.14.6 (Mojave).

So far I was able to get successful connections with IKEv2 and L2TP/IPSec, but all of them use a username/password client authentication. That's not what I want since such authentications are vulnerable by dictionary and brute force attacks. My goal is to use client certificates issued by a self-created certification authority to authenticate the clients.


IKEv2, which is preferred by me to use in the further network configuration, works and the server is able to authenticate itself by a certificate. For client authentication I have to use EAP-MSCHAPv2, because the ZyWall do not supports EAP-TLS. If I do not activate EAP, other clients such Linux strongSwan are able to connect by a certification based authentication but not the MacOS built-in VPN-Client.

After that I tried to use a RADIUS server to authenticate the EAP-TLS request from the MacOS VPN-Client to bypass the not supported EAP-TLS. The RADIUS server successful authenticate the client and give that response to the ZyWall but after that, the ZyWall does not anything with that, so the client get no response and no connection can be established. That behaviour of the ZyWall is the same with the Linux Strongswan VPN-Client. But if the VPN server is a Linux strongSwan too, the RADIUS server EAP-TLS authentication works perfectly.


I also get a woking L2TP/IPSec connection, but I was not able to implement a certificate based authentication for server nor clients as well. Both, the machine authentication and the user authentication, does not work with the certificates. It is only possible to establish that connection while using the PSK for machine authentication and username/password for user authentication. In the ZyWall-logs you can read "Authentication mismatch" and the connection will not be established.


  • ZyWall firmware: 4.33(AAAA.0)C0
    • used as PPPoE access to ISP with dynamic public IP (updated by DDNS from ZyWall)
  • Different MacBooks with MacOS 10.14.6 Mojave and iPhones with iOS 12.4
    • all clients were in a ZyWall independent network at the time of the connection attempts but behind a NAT router of course.
  • freeRADIUS: 3.0.17 (on Raspberry Pi, Raspbian Buster)
    • while using the RADIUS server I have set the authentication server
  • strongSwan: U5.5.1/K4.19.57-v7l+ (on Raspberry Pi, Raspbian Stretch)
  • Certificates issued by OpenSSL
    • Certificates from ROOT CA and Intermediate CA are installed on all machines and marked as trusted, so that verifying the certificates were never a problem
    • ZyWall certificate was created as CSR on the Zywall and signed by the Intermediate CA:
      • CN = hostname.domain.tld
      • X509v3 Key Usage: critical: Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment 
      • X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication, iKEIntermediate
      • All DNS-Names are in the X509v3 Subject Alternative Name listet starting with the DDNS-Name
    • Client certificates are also signed by the same Intermediate CA:
      • CN = first and last name of the user
      • X509v3 Key Usage: critical: Digital Signature, Non Repudiation, Key Encipherment
      • X509v3 Extended Key Usage: TLS Web Client Authentication, E-mail Protection (since the client certificates are also used for S/MIME)
    • At this point I have to say that I tried a lots of different settings for the certificates to rule out that the problems are the certificates itself such as giving TLS Web Server Authentication and iKEIntermediate to the client certificates too and using self-signed root certificates from the ZyWall. In the End I do not think that the certificates are the reason for the authentication problems but I am ready for all ideas.

Since this is the first time I need to use this forum because I am really stuck with that, I hope you can help me. After 4 weeks, I have no idea how to proceed. The last resort is that I continue to use the currently working OpenVPN infrastructure, which I actually wanted to replace with the ZyWall.

Thank you!!!

All Replies

  • Zyxel_Cooldia
    Zyxel_Cooldia Posts: 1,454  Zyxel Employee
    First Anniversary 10 Comments Friend Collector First Answer

    Hi @tomaltmann

    Welcome to Zyxel Community.

    For VPN dynamic rule + IKEv2 + certificate should be doable in this scenario. 

    Did you see any VPN connection fail log in USG?

  • tomaltmann
    tomaltmann Posts: 2
    Friend Collector First Comment
    edited September 2019

    Hallo @Zyxel_Cooldia and thank you for your reply!

    I was able to establish a connection to the ZyWall but only with the Linux Strongswan IKEv2-Client. It was impossible to connect to the ZyWall with the MacOS VPN-Client.

    For MacOS Mojave I used the following settings:

    (Sorry for the "blur", but at the moment I have no test configuration and certs)

    The proposal for the VPN connection where set by a Apple Configurator Profile. The connection attempt will end with a "User Authentication failed" and no connection will be established. The ZyWall give the following output:

    To be sure that the ZyWall settings are not wrong I also tried to connect from the Linux Strongswan VPN-Client with a successful connection. I used the following configuration and get the ZyWall output:

    In both cases I used the same certs and I did not change the ZyWall settings at all. I have added the configuration of the ZyWall's IKEv2 gateway and connection below.

    Thank you for you interest and I am looking forward for you answer!

  • warwickt
    warwickt Posts: 111  Ally Member
    First Anniversary Friend Collector First Answer First Comment

    Hi tomaltmann and @Zyxel_Cooldia I have the same issue above using  MacOS and iOS 13 devices using native VPN clients IKEv2.

    - can't get any thing to work in IKEv2 using certificates nor IkEv1 using certificates ... most frustrated with this.


    failure messages (debug on):

    • Peer IP address mismatch
    • Local Traffic Selector mismatch
    msg="IKEv2 SA [Responder] negotiation failed:" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg=" Local IKE peer 2XX.XXX.14.60:4500 ID (null)" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg=" Remote IKE peer ID (null)" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg=" Message: No proposal chosen (14)" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg=" Reason:" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg="  Peer IP address mismatch" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"
    msg="  Local Traffic Selector mismatch" note="IKE_LOG" user="unknown" devID="1fff0dfec3fff" cat="IKE"


    • Our clients wants remote access to head office facilities for ALL their team ( MacOs and iOS).
    • don't want to expose PSK to rank and file - need to use Certificate / User / Pwd
    • for years only senior selected personal within the client's organisation has L2TP IKEV1 VPN access using a PSK (pre-shared key)
    • this has been works flawlessly for many years to a Zyxel USG110 ?

    Status of failure:

    I've exhausted the obvious and less obvious resources for Zyxel and the others as far back as 2013 to determine how one may get this working including the most obvious .PDFs from a plethora of places ..

    Aside from the above from tomaltmann , many USG110 Debug logs show this frustrating message :

    I've attached the 3 following items for anyone to peruse and assist with advice.

    1. IKE map (IKEV2 phase 1)
    2. Crypto map (phase 2)
    3. debug logging (from syslog host for this test router USG40)

    This iOS VPN client test scenario uses  iPHONE 11 at iOS 13.3.1:

    • Server:
    • Remote Id:
    • Authentication:
      • User Authentication: Certificate
      • Certificate:
    • IKEv2 Certificate.P12 generated from USG40 with Common Name, NR Principal Name and DNS Name as the host name example ""
    •  iOS client (and MacOS 10.10+) uses these crypto
    • Phase 1 : AES256 / SHA256 DH14
    • Phase 2 : AES256 / SHA256 (PFS = none)

    Any advice or assistance is greatly appreciated.



    Hong Kong

  • warwickt
    warwickt Posts: 111  Ally Member
    First Anniversary Friend Collector First Answer First Comment

    Hi guys (those of you interested) tomaltmann @Zyxel_Cooldia, the good news is that we have managed to get this scenario to work and to be most stable indeed.


    The deployment of IKEv2 user/password and same (common) Certificate authentication on USG routers for

    •  MacOS,
    •  iOS,
    • Windows10 OS
    • and Android.

    So far we cannot get or IKEv1 nor IKEV2 Machine Authentication using a common Certificate across the same platforms ?

    • Frankly a few from our team spent some weeks chasing this and frankly it has been very frustrating mainly due to publicly available (inter-webs) clumsy, sometimes overly motherly ,broadly generic and worse blandly incorrect "black magic" type instruction as well as frustrating default configurations and limitation of some VPN IKE inbuilt clients.
    • We have elected at the stage to provision IKEV2 with Certificate , User / Password rather than relying in Machine Authentication with certificate.
    ZYxel IKEv2 Client Deployment documentation: ( grade = 70/100)

    To the credit of Zyxel ?️ there is some good documentation out there to do this, however it falls short when the suggested confirmation doesn't work .. especially when the demonstrable OS platform (usually windows OS) is used.

    • However In particular some from France (in Francais) ??and is very helpful/ readable ...
    • thus one must SCOUR the inter-webs to find these...

    Other documentation: FWIW choose very very carefully ...

    Last Whinge and whining... :

    I would like to see Zyxel publish a detailed list of all their ZYOS messages, their explanations and possible workarounds including Debug messages.

    This should have save our team a greta deal of time.

    Equally , Apple Inc's MacOS has such a plethora of verbose logs spewing forth for its that drives IKEV2 upper with a lesser amount of "red-herring debug messages".. one doesn't expect any thing from them.

    To sum Up:

    To fellow-forum member that experience these frustration with Client-Site IKEv2 please post and SHARE your actual resolutions on these forums.

    Let me know if you need any ideas as we can contribute with what we have learnt on this.


    Hong Kong

Security Highlight