Attention Zyxel lads 我的台湾朋友, this problem reproduceable on USG40 & USG60 with new Firmware update V4.70(AAKY.0)

Scenario

After Upgraded lab USG40 and USG60 lab routers from V4.65(AAKY.0) to V4.70(AAKY.0) over weekend and verification testing we found that........

Problem Issue

All L2TP client connections that require AAA LDAP authentication ( group ldap) are consistently failed in L2TP "authentication failed" on both USG40 and USG60 at Firmware V4.70(AAKY.0)

<local2.debug> usg40 msf-usg40-01 src="222.XX6.1XX.XXX:1701" dst="14.0.XXX.XXX:45716" msg="Message: PPP failure: Reason: Authentication failed" note="L2TP_LOG" user="unknown" devID="1c740dfec31c" cat="L2TP"

Refine:

It is only the L2TP client accounts requiring LDAP PAP authentication that are fail authentication.

• all Local accounts [ the don't rely on AAA LDAP authentication ] in these routers are authenticated against the router's password = work correctly.

Reproduceable:

Yes, on USG40 and USG60 and only at firmware update V4.70(AAKY.0)

Workaround: (hmmm ...)

1. add LDAP users as LOCAL users in router - (works but wont be doing that!)  
2. rollback USG40, USG60 routers to firmware V4.65(AAKY.0)  - then L2TP AAA LDAP works 100% again!. 

Coarse Diags & Conclusion (and suspicion):
V4.70(AAKY.0) firmware for L2TP PPP subcomponent is not processing handoff to AAA LDAP correctly.

Controls:

• LDAP server access available from USG40/ USG60 for both firmware levels : V4.65(AAKY.0) to V4.70(AAKY.0 .. example test LDAP account 'casper' resolves correctly to the external LDAP server from the USG routers themselves:

 Router> test aaa server ldap host ldapserver.lab.studio port 389  base-dn dc=ldapserver,dc=lab,dc=studio bind-dn uid=diradmin_msf,cn=users,dc=ldapserver,dc=lab,dc=studio password XXXXXXXXXXXXXXXXXXXX login-name-attribute uid account casper

resolves correctly to the external LDAP server as:

<div>dn: uid=casper,cn=users,dc=ldapserver,dc=lab,dc=studio</div><div>cn: casper</div><div>uid: casper</div><div>uidNumber: 1027</div><div>sn: casper</div><div>objectClass: person</div><div>objectClass: inetOrgPerson</div><div>objectClass: organizationalPerson</div><div>objectClass: posixAccount</div><div>objectClass: shadowAccount</div><div>objectClass: top</div><div>objectClass: extensibleObject</div><div>gidNumber: 220</div><div>.... snip snip<span>
67122401325669019051132627070243663537895756704320186012999399913688</span><br></div><div>&nbsp;48707 root@LDAPSERVER.LAB.STUDIO:10.0.99.201</div><div>authAuthority: ;Kerberosv5;;casper@LDAPSERVER.LAB.STUDIO;LDAPSERVER.LAB.STUDIO;</div><div>altSecurityIdentities: Kerberos:casper@LDAPSERVER.LAB.STUDIO</div><div>apple-generateduid: 47319A36-323C-4A6A-AFDE-FBF36F2B394A</div><div>description: mocked up LDAP user for LDAP L2TP testing</div><div>loginShell: /bin/bash</div><div>.... snip snip&nbsp;</div><div>homeDirectory: /Network/Servers/LDAPSERVER.LAB.STUDIO/Volumes/ssh_group_home/casper</div><div>Router></div>

see attachments...

Operational Overview - L2TP AAA LDAP and Local Authentication = FAILURE

1. Basic stuff, all L2TP-over-IPSEC VPN connections pass through Phase 1 VPN Gateway then to Phase 2 VPN Connection and the tunnel is successuly built :

msg="Dynamic Tunnel [Client01_L2TP_usg40_GATEWAY:Client01_L2TP_usg40_CONNECTIO:0x0e359c53] built successfully" note="IKE_LOG" user="unknown" devID="1cffffffffff"

msg="Looking up IPSec SA for the L2TP tunnel" note="L2TP_LOG" user="unknown" devID="1cffffffffff" cat="L2TP"
msg="Fetching IPSec SA rule #354" note="L2TP_LOG" user="unknown" devID="1cffffffffff" cat="L2TP" 
msg="Creating outbound L2TP control traffic rule (depends on rule #354)" note="L2TP_LOG" user="unknown" devI
msg="L2TP control rule created: index: #362" note="L2TP_LOG" user="unknown" devID="1cffffffffff" cat="L2TP" 
msg="Tunnel Request Complete" note="L2TP_LOG" user="unknown" devID="1cffffffffff" cat="L2TP" 
msg="[Client01_L2TP_usg40_CONNECTIO(#3)]created incoming IPsec flow, idx: 24981" note="IPSec" user="unknown"

2. .. however as L2TP PAP processing is subsequently invoked that SHOULD be passed to the external LDAP server, 

msg="peer requests authentication protocol PAP" note="L2TP_LOG" .....

3. the LDAP request result (was it even called??) is deemed to have failed authentications as:

<div>msg="Remote L2TP peer 14.0.XXX.XXX:49299" note="L2TP_LOG" user="unknown" devID="1cfff</div><div>msg="Remote tunnel ID&nbsp; &nbsp; 26 session ID&nbsp; 3856" note="L2TP_LOG" user="unknown" devID="1</div><div>msg="&nbsp; User-name: <b>casper</b>" note="IKE_LOG" user="unknown" devID="1cffffffffff" cat="IKE</div>

unauthenticated resulter here----> 

msg="Message: PPP failure: Reason: Authentication failed" note="L2TP_LOG" user="un

 
then the tunnel is destroyed ... etc etc
msg="IPsec SA destroyed: ESP Inbound SPI: [9ee7dec7] Outbound SPI:

Attachments and Diags:
I have attached these logs and listings from the USG40 router .. the same basically for the USG60 router...

• show version_l2tp-over-ipsec_aaa  group server ldap.log
• test aaa server ldap host.log
  
• USG_V4.70-AALA.0_L2TP_IPSEC_PH2_authentication_failure.log
  

.. Lastly

We will not be upgrading to  any other (including clients routers) to Firmware V4.70-AALA.0 until this is resolved due to the reliance of some routers we manager that use L2TP AAA LDAP authencation.

• This circumstance is easy to reproduce. 
• When available will also try on a USG20WVPN and a USG110 at thei respective updated   firmware levels.. ..these will likley be the same and not work wi the above.

Please let me know what other diagnostics you may require of us.

As always and from previous resopnsive and excellent support  back prior to 2018, I look forward to the prompt support from you engineering blokes at Zyxel TW 我的台湾朋友 
 
Kind Regards
Warwick
Hong Kong