Nebula Cloud Service Incident 2023-07-10

Zyxel_Vic

Hi @Pook

Reboot the device may have the chance to recover from current situation since it will reget the settings from Nebula cloud again. However, if it failed, please follow @Zyxel_CSO suggests to share more information of your device and we'll help to check on it.

fraukzsent

I find the following "strange" thing. Site-2-Site-Connection is up —- but not visible in Nebula.

MarkSL

I can confirm with one of my Nebula Site-Site clients that rebooting both ends did allow the connection to return. I have many more to check this morning.

DaviD42

We have rebooted one end but cant reboot the other at the moment. Regardless it would appear that the firewalls have stopped receiving configuration changes from Nebula! Couldn't even reboot it from the console or change the VPN to bypass Nebula!

Is there any news on when we can expect a fix?

Drjo_1

Hi Zyxel - Team,

…when will we get the next update?

Still settings, configuration and monitoring is not possible?

what MTR is expected?

br, jo.

Zyxel_Vic

Hi all

We had finished pushing the settings to all those devices that may have VPN connection issues. Would you please check if it's working right now? If the problem persists, please kindly provide your Org/Site name and Firewall Serial Number by private message for us. We will further assist on your cases.

Thank you and apologized for those inconvenience.

Drjo_1

Hi, I can confirm, that the VPN - Tunnel is working.

Nevertheless the settings cannot (!!) be seen in Nebula. For none of the two sites.

Also the monitor is empty and not showing that the connection is up —- the vpn monitor is an empty page.

br, jo

Drjo_1

https://community.zyxel.com/en/discussion/comment/54538#Comment_54538

Hi DaviD, was your problem resolved? wr, jo.

StefanF

Is there any information on what caused the problems?
Our VPNs were down for about 28 hours causing severe problems.
My superiors will have questions after summer holiday opn what happened and how we can protect us against future problems. Well, answer may be to look at other vendors, or take some steps back and not use cloud based services….

Pook

https://community.zyxel.com/en/discussion/comment/54638#Comment_54638

No such thing as a perfect Vendor or Service that never has a blip or problems. Overall I would say the Uptime for Nebula is excellent and despite the massive headache this caused I wouldn't be interested in other vendors. Like I said at some point or providers will hit a snag, you just have to roll with it.

But ZyXel do need to improve there Service status notifications, a dedicated service webpage would go along way to have helped and more updates. Their tech support is spot on too I will add :-)