Ipsec via main GW stops work, Found old outbound SPI error in debug log

alexey

+1. Today stop works ipsec via USG1100 and USG1000 after reboot of USG1000.
Helps change ipsec gw on other ISP.
USG1100 on 4.65 fw.
We collect debug data via com from 1100 all time. Nothing interesting. It was around 23-45. 

alexey

your forum works like devices...can't attach, qoute, like.

Permission Problem

You need to enable javascript to do that.

On all devices.

alexey

[2021-08-04 20:12:46.404]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 20:13:49.356] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 20:19:04.263] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 20:40:14.787] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 20:51:15.246] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31schedule-run: clear_session_smb.zysh can not be found

[2021-08-04 21:00:28.761] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:04:30.907]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:04:40.597] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:15:44.905]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:15:48.241]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:16:27.783] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:42:23.979]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:42:25.051]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:42:35.164] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:42:43.605]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:42:44.438]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 21:43:54.270] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 22:13:58.698]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=1!!

[2021-08-04 22:14:01.694]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 22:14:10.417] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 22:46:11.793] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 23:07:35.235]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 23:08:19.759] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 23:28:22.163] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-04 23:38:38.118] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:10:25.849]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:10:28.366] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:23:37.877]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:23:38.743]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:23:54.968] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:24:56.520]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:25:28.651] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:30:26.485]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:30:27.696]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:30:32.543]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:30:33.970]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:32:31.606] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:44:12.008] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:59:42.169]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 00:59:48.463]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:01:12.543] unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:04:03.599]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:04:25.958]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:05:49.278] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31  fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:42:44.765]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:42:44.928]   fqdnMODULE_ERR[fqdnObject_process_tcp_dns_data():639] TCP DNS cannot find any matched query node, ret=4!!

[2021-08-05 01:43:29.432] unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31unpack_message_rr fail ret=-31e

Zyxel_Can

Hi @alexey,

Can you share some information with us:

 

1-    Please provide your topology with IP addresses and subnets that related to the sites with the problem.

2-    Please provide IPSec and IKE related logs both for USG1100 and USG1000 to me by private message.(Monitor > Log > View Log > Category : IPSec and IKE)

3- For troubleshooting purposes I would like to ask, is VPN disconnection symptom occurred 4 August 2021 around 23:45?

alexey

The issue returns with bigs steps.
We change old unsupported usg 1100 to new great supported the best ATP 800 some months ago.
FWs on ATP were V5.35(ABIQ.0)ITS-23WK12-0331-230301541 & V5.36(ABIQ.1)ITS-23WK21-r109592 now installed.
On atp around 60 ipsec tunnels to other ZW: 27 to old usg1000, others to ZW110/Flex50w/100.
After update ATP800 to new FW around half of ipsec tunnels don't start. Helps only change Authentication method on both side. If it was certificate, than change on preshared key, Or vice versa.
In logs on both side only exchange with diffs keys on each side.

Zyxel_Jeff

Hello @alexey

Thanks for reporting this case to us. Because the latest firmware has CVE fixes for CVE-2023-33009 and CVE-2023-33010, we strongly suggest you update to our latest V5.36P2 or V4.73P2 firmware.

You can follow the below steps to update firmware:

Step 1: Disconnect the WAN port connection and reboot your device.

Step 2: Access the device's Web-GUI using the LAN port.

Step 3: Update the firmware to our latest version, V5.36P2 or V4.73P2

You can download the firmware from this link: https://portal.myzyxel.com/my/firmwares

Once you update to our latest firmware and it still has a problem, please let us know it.

Thanks.

alexey

https://community.zyxel.com/en/discussion/comment/52668#Comment_52668

🤦‍♂️

I wrote, the issue start after update to V5.36(ABIQ.1)ITS-23WK21-r109592, that = 5.36P2.

So i let you know, that problem continues.

Zyxel_Jeff

Hi @alexey

Could you provide the device Web-GUI and describe the current symptom for us? We will send you a private message later. Thanks.

alexey

Symptoms: after long time ipsec stops working. Help only change Authentication method on both side. If it was certificate, than change on preshared key, or vice versa.

Zyxel_Jeff

https://community.zyxel.com/en/discussion/comment/52649#Comment_52649

Dear @alexey

We noticed your peer site device are ZW110/Flex50w/Flex100, so we strongly suggest you update to V5.36P2 or V4.73P2 firmware due to Buffer overflow might cause the VPN tunnel connection issue, please refer to the below FAQ:

https://community.zyxel.com/en/discussion/17102/buffer-overflow-issue-faq

If you still have a disconnection issue, please collect the syslog and record when VPN is disconnected for us. Thanks for your help.