Zyxel XGS1930 is randomly crashing

Aison

Hi all,

I'm using 10 Zyxel XGS1930 as access switches in our network. The switches are connected per duallink LACP to the core switches. I'm using the newest Firmware V4.7

All switches crash randomly but regularly. There is hardly a week where at least one switch doesn't stop responding. The switches stop forwarding packets and they are also not reachable over their smart management interface. Also ping requests are dropped (on any ports). It is impossible for me to trace for the reason of the crash :-(

All I can do then is powering off/on the switch. Then it works again for some weeks. It is really annoying and is not what I expected of the XGS1930.

Temperature or power problems can be excluded.

Any idea what I can do? I would urgently like to avoid switching to another product.

I appended one of the used configurations. The are almost equal on all switches. Only the vlan id/names and port names differ.

config_XGS1930.log

Zyxel_Kay

Hi @Aison

Based on the configuration provided, it appears that your XGS1930 switch is running firmware version V4.70(ABHT.7). We recommend upgrading to the latest firmware version, V4.70(ABHT.9), which includes important fixes and improvements.

To better understand and resolve this issue, could you please provide the following details?

1. Issue Timeline: Did this problem start occurring from the initial installation, or did it begin after a firmware upgrade? Have there been any configuration changes in your network setup?
2. Frequency of Crashes: How long does the system remain operational after a reboot before it crashes again?
3. Technical Support File: Could you collect the tech support file from the affected switch? This file will help us analyze the logs and identify the root cause.

Aison

Hello Kay

Thanks for your fast answer. I didn't notice the change from V4.70(ABHT.7) to V4.70(ABHT.9). I upgraded now all switches to ABHT.9 and I'm going to monitor the situation.

1. It started from the initial installation. The whole installation is new (installed this summer)
2. Hard to say. It's not regular. Sometimes it takes just a few days, sometimes it may take weeks.
3. I appended the tech support file.
   techSupport_all.zip

Zyxel_Kay

Hi @Aison

From our analysis, it appears that your switch is not experiencing a crash in the conventional sense. Typically, when a crash occurs, the switch would automatically reboot itself. Instead, the issue you’re facing seems to be related to high CPU usage, which is causing the switch to stop responding.

After reviewing the logs, we observed multiple LLDP notifications on the switch. These notifications are leading to high CPU usage and may be contributing to the problem.

We recommend disabling the following features on all Zyxel switches in your network and monitoring the results:

• LLDP MED topology-change-notification
• LLDP notification

If the issue persists after making these changes, could you please:

1. Collect and share the updated tech support files from the affected switches.
2. Provide your network topology, including details of how the switches are connected.

This additional information will help us perform a deeper investigation.

Aison

Hello,

Thx for the answer. I disabled now all LLDP notification on all Zyxel Switches. LLDP MED was not in use anyway. But maybe there are other services like lldpd (from linux servers) sending these kind of notifications?!?

In the meantime, another switch of same model was not responding. I guess it's also related to LLDP message flooding. But on this device, I can't download the tech support files "all", since it results in a timeout on my browser?! I appended the other tech files instead.

techSupport_L3.zip

techSupport_mbuf.zip

techSupport_memory.zip

techSupport_crash.zip

techSupport_cpu.zip

Zyxel_Kay

Hi @Aison

Thank you for sharing the information and technical support files.

To assist with further troubleshooting, we would appreciate your help with the following:

1. Network Topology:
   • Please provide a detailed network topology. A diagram would be especially helpful.
2. LLDP Feature:
   • If the LLDP feature is not in use, could you disable LLDP on the switches involved?
3. Tech Support Files:
   • Select a switch experiencing the issue most frequently to collect technical support files.
   • Collect the tech support file every three days so we can compare the differences between diagnostic information.
   • If the issue occurs within this three-day period, please collect another tech support file immediately after the issue.
   • For the tech support file collection process:
     • Attempt to collect the file via the web GUI.
     • If the web GUI fails, please use an SSH client (e.g., Tera Term or PuTTY) and run the CLI command show tech-support.
4. Console Cable:
   • If you have a 4-pin console cable, please let us know. Since the device hanging occurs randomly, we may need your help to collect debug logs when the issue arises.

Thank you for your cooperation.