XS1930-10 New Firmware Problem 4.80(ABQE.4)C0

Av3ng3r

I just upgraded the firmware on 2 XS1930-10 switches to 4.80(ABQE.4)C0 and noticed high and constantly high CPU utilization for my configuration. CPU steady above 36%.

My switches only have 3x 10 gbps connection and 1x 2.5 gbps and the other one has a 2x LAG 10gbps to a NAS, 2x10 gbps to a computer and AP.

I reverted the firmware to the previous version: 4.80(ABQE.3)C0, and CPU is back to 6% utilization.

Anyone experiencing/experienced the same problem?

Zyxel_Kay

Hi @Av3ng3r

With the firmware update to 4.80(ABQE.4)C0 for the XS1930 series, we included an enhancement for switch port recovery. This new mechanism periodically checks for any ports that may have become unresponsive or lost connectivity. When a port issue is detected, the switch works to restore it automatically, which requires some additional CPU resources.

This enhancement addresses a previously identified issue outlined in the release notes (bug fix #1), as it prevents ports from remaining inactive without user intervention. You can find more details in the release notes here:

XS1930-12HP_4.80(ABQF.4)C0_2.pdf

Zyxel_Kay

Hi @Av3ng3r

With the firmware update to 4.80(ABQE.4)C0 for the XS1930 series, we included an enhancement for switch port recovery. This new mechanism periodically checks for any ports that may have become unresponsive or lost connectivity. When a port issue is detected, the switch works to restore it automatically, which requires some additional CPU resources.

This enhancement addresses a previously identified issue outlined in the release notes (bug fix #1), as it prevents ports from remaining inactive without user intervention. You can find more details in the release notes here:

XS1930-12HP_4.80(ABQF.4)C0_2.pdf

Av3ng3r

Hi Zyxel_Kay,

Thank you for your prompt reply.

In my particular case, going from 6% to above 36% seems like a 600% increase for a very light configuration. I saw the CPU even hitting 40% steadily. This translates into additional heat and power consumption.

600% increase seems too much for just checking port status only.

Is there a way to upgrade and disable this feature/function?

cbeckstein

I know this is a thread that adresses the XS1930, nevertheless..

the enhancement for switch port recovery of the XS1930 that https://community.zyxel.com/en/profile/Zyxel_Kay mentioned above is very interesting

could it possibly be extended to the XMG1915E and fix a similar recovery problem which I brought up here

https://community.zyxel.com/en/discussion/21800/reboot-of-rstp-master-causes-continued-inaccessibility-of-rstp-slave-2-xmg1915-10es

This problem is still unfixed in the April 2024 FW for the XNG1915E (you were not able to repropduce it) and believe me, it is a pain in the a****

Zyxel_Kay

Hi @Av3ng3r

Thank you for sharing your feedback.

Our team is actively working to optimize the port detection process to reduce CPU usage, and we plan to include this improvement in an upcoming firmware release.

For the latest updates, please keep an eye on our https://community.zyxel.com/en/categories/switch-news-and-release . We appreciate your patience and will keep you informed.

Hi @cbeckstein

After reviewing the forum thread you referenced, I can confirm that your case is completely different. The switch port recovery enhancement on the XS1930-10 specifically checks for ports that may have become unresponsive or lost connectivity and automatically restores them when such issues are detected. In your case, however, the traffic is functioning; the issue is limited to your inability to access the switch. Additionally, we have not received any reports of unresponsive or lost connectivity issues on the XMG1915 switch.

If you need further assistance, please provide more details on that forum thread so that our team or other users with similar experiences can better assist you.

cbeckstein

Sorry, no,…

In my case the slave XMG1915 is NOT only inaccessible (for ping or login), there is no more traffic from the master XMG1915 to it - until the next reboot of the slave

So I dare to contradict your claim that the cases are completely different

The slave does not handle correctly / recognize its master going down and up again and along wioth all the network devices connected to it - gets lost

Zyxel_Kay

Hi @cbeckstein

Could you please provide our colleague with the packet file she requested earlier? This will help her further investigate your issue.

Kristof

https://community.zyxel.com/en/discussion/comment/71109#Comment_71109

Hi @Zyxel_Kay,

I am having exactly the same issue with my XS1930.

The CPU increase is unbelievable.

And I am having the same question as @Av3ng3r, will there be an option to disable this?

As in my configuration I will not need it anyway.

I have also reverted back to a previous firmware, the CPU increase is just too high.

So I hope there will be a new firmware with a proper fix soon.

Thanks for any information you could provide.

cbeckstein

@ https://community.zyxel.com/en/profile/Zyxel_Kay

Sorry no, I cannot do that:

I already spent many hours in March and April chasing bugs in the XNG1915 (resulting in 4 fixes for the April 2024 FW) but at the moment I simply cannot afford the time for lengthy experiments with wireshark stuff

Zyxel_Kay

Hi @Kristof

Regarding this case, our team is actively working to optimize the port detection process to reduce CPU usage, and we plan to include this improvement in an upcoming firmware release.

For the latest updates, please keep an eye on our  https://community.zyxel.com/en/categories/switch-news-and-release .

Zyxel_Kay

Hi @cbeckstein

Regarding your previous post, it seems that your switch usually recovers after a reboot. However, with the XS1930, if a port experiences unresponsiveness or loses connectivity, a reboot alone won’t resolve it; additional steps are needed to debug. The XS1930 enhancement aims to address this bug, which is why I mentioned that it's different from your case.

To identify the root cause of the issue on your XMG1915-10E, we need some additional information. Previously, we requested a packet capture with all end devices disconnected from both switches, which may have caused some network interruption. If you'd prefer, we can try an alternative approach that doesn't require disconnecting any devices.

Please mirror the packets from both switches' uplink ports for 10 minutes and share the capture file with us. We'll use the file to replicate the issue in our lab environment. Since ports 9 on both AZ and WZ switches are currently unused, you can connect a laptop to one of these ports to serve as the monitoring port. To ensure easier handling, please split the packet capture into files of 200 MB each.

Once you've completed this, kindly send us the packet files through a private message.

Looking forward to your response so we can help resolve this issue.