It has been over year of this issue

PeterUK

And I I'm on the edge of losing it!

Routing Zywall out a given interface when you have other WAN In my care my main virgin media and EE 5G and O2 G4 as backup

Still in talks with Salim about this.

Here is what we know if you don't do routing of Zywall all works fine if you set the default trunk to just one interface with routing of Zywall to that interface that works.

So wait if I can have the Zywall connect fine to my main virgin media with the above why it there a problem!

Like I want to route Zywall for its updates out a given interface and such yet when I do a Network Tool Nebula status check it times out yet there is a session from given source port to the Nebule for its Connectivity yet a simple SYN which does leave my network I get no SYN ACK!

Yet sometimes if you try Network Tool Nebula status check over and over it will succeed and by changing the truck back and too along with disable/enable backup WAN seem to cause the issue to happen.

SO here is what I think is going on...someone over engineered the Nebula firewall I mean what I'm about to say is just nuts! But it can only be the way they have done it which is they are using TCP timestamps as a type of SYN Authentication! So here is what might be happening the USG some how connects to the server and gets many random timestamps for use for sending a SYN the server keeps track of but also based on the default trunk but because I route Zywall out a given interface it uses a timestamp meant for another interface WAN so the SYN gets to the server and drops it because the source IP and timestamp don't match.

Zyxel_Melen

Hi @PeterUK

To better investigate your issue, I use AI to summary this issue to understand the current situation. Could you help to check if the summary match?

ZyWALL Multi-WAN Routing & Nebula Connectivity Issue Summary

Environment Setup

WAN Connection	Purpose
Virgin Media	Primary
EE 5G	Backup
O2 4G	Backup

Observed Behavior

What works:

• No routing policy for ZyWALL → Everything works fine
• Default trunk set to a single interface + ZyWALL routed to that same interface → Works fine

What doesn't work:

• When ZyWALL traffic is routed to a specific interface (for updates, etc.):
  • Network Tool → Nebula status check times out
  • Session is established (source port → Nebula server)
  • SYN packet does leave the network, but no SYN-ACK is received
  • Repeated attempts occasionally succeed
  • Toggling trunk settings or enabling/disabling backup WAN seems to trigger the issue

Your Theory

Nebula's firewall may be using TCP Timestamps as a form of SYN authentication

Hypothesized mechanism:

1. The USG connects to the Nebula server and receives multiple random timestamps for future SYN packets
2. These timestamps are associated with specific WAN interfaces (i.e., source IPs)
3. When ZyWALL traffic is policy-routed to a different interface, the SYN uses a timestamp intended for another interface/IP
4. Nebula server detects the mismatch between source IP and timestamp → drops the SYN

If above are correct, are there any additional details or behaviors I missed?

Have you captured any packet dumps that could help verify the TCP timestamp theory?

Also, in order to replicate this issue, could you share which DMZ type is this case and share your configuration with us? Also, we need the traffic flow in this DMZ topology so we can better investigate this issue.

PeterUK

If you lookup case #446011 you will find the analysis of my packet dumps your Hypothesized mechanism is in line with what I'm thinking.

So in testing is FLEX200H with VLAN443 and EE 5G backup VLAN31 I route incoming Zywall with *zyxel.com and *myzyxel.com next hop VLAN443 which links up to FLEX200 SNAT my Virgin Media.

Whats also happening is there is must be a mechanism should your IP change not sure hows that done It could be a given TCP timestamp must be accepted from any source IP at the Nebula server and their looks to be a override of the routing rule where Zywall goes out external interfaces for a whats my IP to the Nebula server every 10 minute so it knows and FLEX knows if behind NAT or CGNAT

so what I think is happening is FLEX200H uses a TCP timestamp meant for EE 5G backup but the routeing rule overrides that and goes out VLAN443 and because the IP has not changed the FLEX does not use its given TCP timestamp from any source IP but at times FLEX does use the correct TCP timestamp.

There a another problem with this system if thats how it works but thats not the case here.

So if what I said is true there are two ways to fix the problem

1. Is that there are expected timestamps for one given IP to follow vs another so group them to allow from either from a given FLEX unit.

2. is to code the FLEX to understand if their is a routing rule for Zywall so that it use the correct TCP timestamp.

Zyxel_Melen

Hi @PeterUK

Our team has reviewed this case again and carefully clarify this issue, I will keep you posted once we have further information.

PeterUK

Yes so I read by Salim hope something can be done about it

I also give them this scenario which might be a problem

FLEX H WAN1 > other router with WAN1 and WAN2 in a non fail over setup

In this setup WAN1 of the FLEX H knows nothing about the router upstream and its connections can change from it outgoing source IP by other router to Nebula so would this be a problem you can't easily solve?

Mk88_it

Hello @PeterUK , first of all, thank you for your valuable work and contribution to the community.

I think we have the same behavior in some similar customers' configurations…

We will try to run some tests about this and post the results in this thread

PeterUK

Well I tried they are not going to fix it.

PeterUK

Also I small test I did mean they can't be doing this by TCP Timestamps unless they make exceptions for given ISP that do CGNAT how I test was EE and O2 to my web server on VM when using EE they keep the Sequence Number and timestamp the same but change the source port however O2 thinks we change and randomize all three out there CGNAT and like why!

So looks like time windows per interface that Nebula expects to see your USG on which is fine if you have one WAN source because it wouldn't do that!