Forced Firmware Rollout

Pook
Pook Posts: 143  Ally Member
First Comment First Answer Friend Collector Nebula Gratitude

Anyone else seeing the appliances being upgraded this morning?

All Replies

  • Dylan96
    Dylan96 Posts: 21  Freshman Member
    Zyxel Certified Network Administrator - Nebula Zyxel Certified Network Administrator - Security First Comment Friend Collector

    YES, our firewall and many of our client's firewalls got restarted and upgraded without ANY notice and OUTSIDE our configured schedule. What is going on?

  • Pook
    Pook Posts: 143  Ally Member
    First Comment First Answer Friend Collector Nebula Gratitude

    As per https://support.zyxel.eu/hc/en-us/articles/11616709217810I was expecting there to be an update today, but not forced without notice!

  • Zyxel_Melen
    Zyxel_Melen Posts: 2,422  Zyxel Employee
    Zyxel Certified Network Engineer Level 1 - Switch Zyxel Certified Network Administrator - Switch Zyxel Certified Network Administrator - Nebula Zyxel Certified Sales Associate

    Hi @Pook & @Dylan96,

    We released the V5.36P2 firmware. But we didn't set up a forced upgrade.

    Could you help to check the firmware version? If is V5.36P2, please help to enable Zyxel support and provide the org & site name. We will help to check the log.

    Apologize for any inconvenience caused.

  • Pook
    Pook Posts: 143  Ally Member
    First Comment First Answer Friend Collector Nebula Gratitude

    Hi Melen

    I can confirm that is the firmware that has been rolled out across multiple Orgs we mange this morning. All sites are back online. Message me for info on an Org you can review.

  • Had same scenario here, where two devices force upgraded this morning.

    The force upgrade window as a whole is pretty short(6/8), if there is a mitigation available, I'd like to see this extended. Forced upgrades outside of maintenance windows are tough to schedule for 24/7 operations.

  • Pook
    Pook Posts: 143  Ally Member
    First Comment First Answer Friend Collector Nebula Gratitude
    edited May 2023

    To make matters worse we are now facing issues of Firewalls not coming back online after applying Firmware (All Nebula mode Flex 200). Trying a factory reset now in the hopes we can get them back up and running.

  • Zyxel_Melen
    Zyxel_Melen Posts: 2,422  Zyxel Employee
    Zyxel Certified Network Engineer Level 1 - Switch Zyxel Certified Network Administrator - Switch Zyxel Certified Network Administrator - Nebula Zyxel Certified Sales Associate
    edited May 2023

    Hi @Pook, @Dylan96, and @MediaManaged,

    Thanks for bringing this to our attention.

    Upon further examination, it has come to our attention that an error in our operations led to the accidental upgrade of your firewall. We sincerely apologize for any inconvenience this may have caused you.

    If you encounter any problems after upgrading, please feel free to contact us.

  • Pook
    Pook Posts: 143  Ally Member
    First Comment First Answer Friend Collector Nebula Gratitude

    Hi Melen, we are having many issues with appliances no longer booting with sys light flashing. I am currently on the Teams meeting with Tobias and he is going to send me the recovery procedures. But if you have access to them feel free to post here :-)

  • Dylan96
    Dylan96 Posts: 21  Freshman Member
    Zyxel Certified Network Administrator - Nebula Zyxel Certified Network Administrator - Security First Comment Friend Collector

    Good morning @Zyxel_Melen

    Thank you for your time, i hope you understand this is unacceptable. We had 3 other firewalls that got auto updated, resulting in an unexpected downtime for our clients…

    We hope this never happens again…

  • Zyxel_Melen
    Zyxel_Melen Posts: 2,422  Zyxel Employee
    Zyxel Certified Network Engineer Level 1 - Switch Zyxel Certified Network Administrator - Switch Zyxel Certified Network Administrator - Nebula Zyxel Certified Sales Associate

    Hi @Dylan96,

    We apologize for any inconvenience this may have caused. Please rest assured that we will take extra care with this particular aspect in the future.

Nebula Tips & Tricks