MTU size on SYN packets

PeterUK

I have been trying to get my head round this problem and think this could be solution.

As it is the MTU only apply to SYN, ACK by modifying the MSS in packet which tells the client how big the packet to send to the remote end. The problem is the client sending SYN the with MSS 1460 needs to be modified or else the remote end sends MTU 1500 to you for your tunnel set at MTU 1476 and USG sends ICMP Destination unreachable (Fragmentation needed).

And another problem is going from interface MTU 1500 to tunnel MTU 1476 to interface MTU 1500 so when you receive the SYN, ACK from tunnel the interface set it back to MSS 1460 so when the client send to the remote end its to big.  

Edit more testing.

So only the interface for MSS SYN, ACK is changed the tunnel does not touch the SYN, ACK but does let the interface know by ICMP Destination unreachable (Fragmentation needed) to the client.

So in order to stop ICMP Destination unreachable (Fragmentation needed) the tunnel needs to change the MSS to 1436 (if I got that number right) for both SYN and SYN, ACK then the interface must not increase it even if its MTU is higher. 

tonygibbs16

Hello @PeterUK

I have had an experience where an IPSEC tunnel can only send less than 1500 bytes as MTU (1476 bytes I think) and we had an issue where a conversation was trying to sent 1500 byte MTU over it.

The solution we found was to make the conversations (interface in your example) using the tunnel use the same MTU as the tunnel itself.

      - On Windows we found a registry key value that needed to be set (for interface in your example) to have the MTU match the tunnel.

                   - PureVPN have this explanation at https://support.purevpn.com/how-to-change-mtu-value-on-windows-xp-vista-7-and-8 

Thoughts?

Kind regards,

      Tony

PeterUK

But what if its a device that you can't set the MTU?

Setting the interface MTU 1500 to match the tunnel MTU 1476 only help the client not send 1500 MTU to the remote end not the remote end send smaller packets to the client so the solution I posted will help send smaller packets in both directions.

tonygibbs16

Hello @PeterUK

I think that a registry type out of band setting can be done at either end of the users of the tunnels, which is what my colleagues and I have done at work.

How would your solution work? Are you suggesting that the routers either end know the MTU to use? Is this supported in the TCP RFCs?

Or are you proposing Path MTU Discovery? see https://en.wikipedia.org/wiki/Path_MTU_Discovery and https://datatracker.ietf.org/doc/html/rfc1191

     - Path MTU Discovery sounds promising and it works with IPv4 and IPv6, where the MSS is reduced in size in response to ICMP Fragmentation Needed response (Type 3 Code 4) specifying the MTU supported.

Thoughts?

Kind regards,

     Tony

PeterUK

All the USG need to do when packets go down or out of a GRE tunnel is modify the MSS for SYN and SYN , ACK packets so if the MTU of the tunnel is 1476 the MSS needs to be 1436 to stop for TCP at least ICMP Destination unreachable (Fragmentation needed) UDP can be sent as fragmented when it hits the tunnel.

Currently if the remote end does send a full MTU packet the USG send out a ICMP telling the remote end MTU of next hop: 1476 but what if the remote end blocks this then you get no connection unless the server or TCP on the other end thinks well I sent a full 1500 MTU and got no ACK? maybe I need to sent a smaller MTU and some sites do this but some don't.

setup

tonygibbs16

Hello @PeterUK

As I don't work for Zyxel, I will let them decide what to think about your idea.

I guess there would have to be additional configuration settings to make the USG use a smaller MSS/MTU...

But I think that there could still be issues, because the USG is also acting as a router (an OSI-RM layer 3 function or at IP layer) so does not the solution need to work in that situation too? When a host other than USG is initiating the TCP conversation?

       - so the hosts setting up TCP conversations through the USG will also need to find out that the tunnel has a smaller MTU than 1500 bytes.

I agree with you that ICMP is sometimes/often blocked (e.g. for security reasons), which would make the Path MTU Discovery not work.

In which scenario, an out-of-band solution would be needed to tell the hosts using the tunnel to use a smaller MTU.

Kind regards,

     Tony

PeterUK

Both ends will know what MTU to use as TCP SYN says this is the MSS I can receive so the remote end sending the TCP SYN knows what size packet to send which the tunnel changes in flight for MSS then the client receives the TCP SYN, ACK and the tunnel changes this the MSS in flight but if it happens to be lower then no change.

So what if the client was the one receiving the TCP SYN then same thing the packet in flight would change the MSS to what the tunnel can fit or if lower then no change the TCP SYN, ACK would then be changed for the tunnel in flight of the smaller MSS.

tonygibbs16

Hello @PeterUK

I think that I humbly disagree with you.

Once the tunnel is up, such as a GRE, then the USG cannot affect the MSS/MTU of TCP converations being routed through it, except by saying that they are too big or fragmenting the segments if allowed.

Hosts setting up TCP conversations via the tunnel would have to know the MSS/MTU of the tunnel, e.g. by obeying Path MTU Discovery or out-of-band configuration.

If a Host with a TCP conversation is sending segments flagged as don't fragment that are larger than the tunnel can support, then all that the USG or any tunnel endpoint device (from any manufacturer) could do IMHO would be to send an ICMP Fragmentation Needed response (Type 3 Code 4) specifying the MTU supported.

If the Hosts setting up the TCP conversations did not receive or obey the ICMP response, then the TCP conversation would not pass segments with MSS/MTU larger than the tunnel can pass.

Kind regards,

     Tony

tonygibbs16

Hello @PeterUK

https://support.microsoft.com/en-us/topic/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than-576-2312050e-2a1b-1e89-94a1-b2868afa436f is what Microsoft said for Windows Server 2003 for this situation.

Kind regards,

     Tony

PeterUK

Of course the USG can affect the MSS/MTU of TCP SYN and SYN, ACK it can do it before it goes down tunnel then it gives that packet to the tunnel.

Easy

And again what If the device has no way of setting the MTU like a phone  

Maybe your missing the point the the SYN and SYN, ACK tell each other in the packet how big to send each other packets so that it fits down the tunnel?  

tonygibbs16

Hello @PeterUK

I might have misunderstood, but I think that a protocol stack diagram would be as follows:

If this is correct, then the TCP conversation and therefore the MSS/MTU negotiation would be between the Client and Server, and not with the USG.

     - so the USG cannot change the MSS/MTU once the tunnel is established, except to say the segments are too big.

Therefore not easy I think for the USG to change the MSS/MTU once the tunnel is established, because as a router it would not be engaging TCP.

If a phone cannot set the MTU size, then that is an issue for the phone. It could implement Path MTU Discovery.

      - https://stackoverflow.com/questions/5264344/is-there-a-way-to-change-android-phones-mtu-size/7847980#7847980 says how to change MTU of an Android phone by rooting it.

      - https://android.googlesource.com/kernel/msm/+/android-msm-shamu-3.10-n-preview-4/Documentation/networking/ip-sysctl.txt says how to set a flag in Android to enable / disable Path MTU Discovery.

Thoughts?

Kind regards,

      Tony