GS2220 Firmware V4.80(ABRR.0) | 01/18/2023 - NTP issue

Options
henriquev
henriquev Posts: 24 image  Freshman Member
First Comment Friend Collector First Anniversary
edited April 22 in Switch

Hello,

We have for some time now dealing with an issue where some switches do not properly register to Nebula even after a factory reset.

After some investigation, I believe the issue lies on the NTP packet that the switch sends to "time.google.com" and it is discarded due to the field "Transmit Timestamp" being all zeros. At least in my packet capture sessions the switch didnt try to reach for *.pool.ntp.org, only google.

Update: After firmware upgrade to latest, the "transmit timestamp" is still all zeros, but it doesnt look to be a problem to google anymore and the server replies. Regardless, the switch now queries on pool.ntp.org as well. DM me if you need a capture to look into this issue.

The above assumption was incorrect, in the particular site google still didnt reply, but in the latest release the switch promptly attempts pool.ntp.org as well. And the issue lies where the switch 4.80 waits too long before try to reach another NTP service, it took 4 days of failure trying time.google.com, to switch to try for pool.ntp.org.

Often the switch comes back to Nebula but I cannot do a SSH session, for instance, problaby due to certificate/time issue related to the NTP time sync fail - in cases where the switch eventually registers to Nebula I can see several logs related to time sync fail.

I have seen that there is fixes related to NTP on recent release notes and I assume it is to solve this kind of issue, but the "V4.80(ABRR.0) | 01/18/2023" is still marked on Nebula as stable, and as NTP seems to be critical to a proper registration, it would be nice if a release with this issue fixed is promoted to stable.

Accepted Solution

  • Zyxel_Judy
    Zyxel_Judy Posts: 2,427 image  Zyxel Employee
    Zyxel Certified Network Engineer Level 2 - Nebula Zyxel Certified Network Engineer Level 2 - Switch Zyxel Certified Network Engineer Level 2 - Security Zyxel Certified Network Engineer Level 1 - Nebula
    Answer ✓

    Hi @henriquev ,

    Thank you for sharing the packet capture.

    As you mentioned, your GS2220 is now successfully connected to Nebula with the latest firmware. In recent firmware versions, we have expanded the NTP server list from 1 to 3, which may have addressed the initial issue seen in V4.80. Regarding the stable firmware version for this model, we will evaluate.

    As for the 'Transmit Timestamp' being all zeros — this is expected behavior, as the switch acts as an NTP client sending a time request, and the Transmit Timestamp field is typically not populated by the client. It is the NTP server that fills in the timestamp in its response. This behavior does not affect the switch's ability to connect to Nebula.

    Zyxel_Judy

All Replies

  • Zyxel_Judy
    Zyxel_Judy Posts: 2,427 image  Zyxel Employee
    Zyxel Certified Network Engineer Level 2 - Nebula Zyxel Certified Network Engineer Level 2 - Switch Zyxel Certified Network Engineer Level 2 - Security Zyxel Certified Network Engineer Level 1 - Nebula

    Hi @henriquev ,

    We're glad to hear your issue has been resolved with the latest firmware update.

    Regarding your mention of "After firmware upgrade to latest, the "transmit timestamp" is still all zeros,", could you please provide some additional details:

    • What did you mean by that?
    • How did you test it / perform the packet capture? Please share the results or a screenshot.
    • Did it have any impact on your network service?

    Zyxel_Judy

  • henriquev
    henriquev Posts: 24 image  Freshman Member
    First Comment Friend Collector First Anniversary
    edited April 21

    Hi Judy,

    I will share my packet captures directly into your inbox as it has somewhat "sensitive" information.

    I have captured using a Zyxel Flex 100HP.

    The impact is a non registered switch is unmanaged and cannot apply configuration, etc.

    In the end the affected switch end up having its NTP being from *.pool.ntp.org, as It for any reason could not sync with time.google.com. (it could be a service provider issue, or any incompatibility between zyxel and the pool of servers it was trying to NTP sync with?!)

    From what I could gather the issue is this version of ZyNOS (4.80) takes too long to try to sync to pool.ntp.org, it tries literally for days on time.google.com before actually moving on. Meanwhile, at least version V5.00(ABRR.3) it instatly tries for pool.ntp.org if time.google.com do not succeed. I dont know what is the behavior on V4.80(ABRP.1) | 06/13/2024.


    Output below from a faulty switch, after 4 days (give it ou take it) it managed to try and sync with pool.ntp.org instead of time.google.com that for some reason could not do.

    Switch# show timesync
    Time Configuration
    Time Zone :UTC 0
    Time
    Sync Mode :USE_NTP
    Time Server IP Address :pool.ntp.org
    Time Server Sync Status:OK
    Time Server Sync Interval(minutes):1440
    Switch# show version
    Current
    ZyNOS version : V4.80(ABRR.0) | 01/18/2023
    Image 1
    ZyNOS version : V4.80(ABRR.0) | 01/18/2023
    Image 2
    ZyNOS version : V4.80(ABRR.0) | 01/18/2023


  • Zyxel_Judy
    Zyxel_Judy Posts: 2,427 image  Zyxel Employee
    Zyxel Certified Network Engineer Level 2 - Nebula Zyxel Certified Network Engineer Level 2 - Switch Zyxel Certified Network Engineer Level 2 - Security Zyxel Certified Network Engineer Level 1 - Nebula
    Answer ✓

    Hi @henriquev ,

    Thank you for sharing the packet capture.

    As you mentioned, your GS2220 is now successfully connected to Nebula with the latest firmware. In recent firmware versions, we have expanded the NTP server list from 1 to 3, which may have addressed the initial issue seen in V4.80. Regarding the stable firmware version for this model, we will evaluate.

    As for the 'Transmit Timestamp' being all zeros — this is expected behavior, as the switch acts as an NTP client sending a time request, and the Transmit Timestamp field is typically not populated by the client. It is the NTP server that fills in the timestamp in its response. This behavior does not affect the switch's ability to connect to Nebula.

    Zyxel_Judy

  • henriquev
    henriquev Posts: 24 image  Freshman Member
    First Comment Friend Collector First Anniversary

    Thanks for the update Judy!

    Regarding the "transmit timestamp" it was an assumption as in my research it could be one of the reasons that a server could not reply.