Possible bug when doing a group WILDCARD FQDN list

PeterUK

So I want to limit a PC to what sites it can go to (this incudes other URL the pulls) so I started a list by doing WILDCARD FQDN like *.bbc.co.uk, *.bbci.co.uk and so on but as I started adding to the group list and I can see the DNS requests go to the USG the list stopped working for some new sites and showed as blocked and not listed in the IPv4 FQDN Object Cache List. So I redid the list this time be adding WILDCARD FQDN as *bbc.co.uk, *bbci.co.uk and so far no problems. The problem looks to be doing *. and not * without the dot.

PeterUK

Ok some what a bigger problem I have a big list of WILDCARD FQDN in a group and nothing stopped me adding this many and now sites which I have not listed are allowed by ports 80 or 443. So removed the Policy Control rule and removed the group WILDCARD FQDN and there is no other Policy Control rule to allow ports 80 or 443 and I can still access any site!

Edit: I regrouped the FQDN in groups of 10 and rebooted seems to of helped.

maybe their is IPv4 FQDN Object Cache List limit? I'm at over 250 listed and some sites that are listed are not being allowed.

So I have the following for amazon.co.uk in WILDCARD FQDN

*amazon.co.uk

*amazon.com

*amazonaws.com

*amazon-adsystem.com

*aiv-cdn.net

*aiv-delivery.net

*aboutamazon.co.uk

*acx.com

*ssl-images-amazon_com

*media-amazon_com

Yet the site will not load other sites I listed work I can see DNS go to the USG but its not its just not allowing it yet I know if I reboot the USG amazon.co.uk will strat working again. So problem could  be when it first lists amazon.co.uk on the list and works over time amazon.co.uk IP's change by DNS and the USG is not updating the list to allow access?   

PeterUK

update

So I did each WILDCARD FQDN to a Policy Control rule which took some time but the result was sites load much better and was able to play a film from amazon without interruption some times when the browser has not close for some time amazon page don't load but a close/open of the browser fixes that.

So their really does seem to be a issue with WILDCARD FQDN in a group.

So...reboot make it works I add more WILDCARD FQDN to my list some sites stop loading reboot it works a bit...or is it just the  IPv4 FQDN Object Cache List is fresh and when it gets to 350 it starts going wrong again...Im going to not add more WILDCARD FQDN to the list and see how stable it is. 

PeterUK

Well I think I found some of the problem and that is I have another interface card set with another DNS that don't go through the given USG and some of the DNS was going out one and the other so the given USG didn't know what the device was looking up.

But that still don't explain the big list of WILDCARD FQDN in a group and allows any site not listed.

Zyxel_Stanley

Hi @PeterUK

As your description, there are 2 NIC installed on your PC. Are both NIC behind the USG?

What if you disable one of NIC and test FQDN group object again?

Just tested by similar conditions (multiple FQDN rules) but we didn’t what the symptom happened on your description. If your symptom still exist, you can provide your configuration by private message.

PeterUK

Yes my bad for testing this as one NIC with no gateway set but has a DNS set 192.168.255.55 does not go to the same USG as another NIC with no gateway set with DNS set 192.168.53.4 and a another NIC for WAN that goes by that same USG.

Testing by removing 192.168.255.55 from the NIC now works well by many  WILDCARD FQDN added one by one for Policy Control rules.

If you do a group of say 100 WILDCARD FQDN by HTTP/HTTPS with no other allowed HTTP/HTTPS test all the sites in your list then test a site not on your  list does it allow it? As thats one of the problems I saw with a big group and it allowed any HTTP/HTTPS even with all the rule disable!

edit not sure but its working now for a group with 128 in one group and I did work out that some sites share the same IP so maybe adding to a group one at a time and testing could be a issue over time then adding 128 in one go? not sure...    

thanks  

PeterUK

@Zyxel_Stanley

After a week of uptime with the USG60 with my two groups of WILDCARD FQDN its now being blocked nothing in logs saying its blocked just will not load any site now. whats even more odd is I try to do a DMZ to WAN allow HTTP/HTTPS and its still blocked! I then tried DNS to 8.8.8.8 in case it was my DNS setup and found some thing really odd! I have a Wireshark after the USG and on the PC and for some reason when nslookup for bbc.co.uk by 8.8.8.8 the request goes out the reply goes to USG 60 but the USG 60 does not forward to the PC for IPv4 only IPv6! So it looks like the passive DNS sniffer broke in such a way that blocked DNS replies. Only option was to reboot the USG.

I did a collect Diagnostic Information before rebooting should you need that let me know.