break HTTPS Domain Filter for HTTPS traffic

PeterUK

Zywall 110 V4.72(AAAA.0)ITS-22WK28-r104687

I seem to have found a way to break HTTPS Domain Filter for HTTPS traffic when editing the zywall 110 content filter and needing to be rebooted to fix it.

So heres what I'm doing the HTTPS Domain Filter for HTTPS traffic is broken (not sure how yet I hope I can make it happen again) I have a Profile grc3 with custom service tab enable Custom Service and Check Common Trusted/Forbidden List checked in Forbidden Web Sites I have *.grc*.com then in policy control top rule with grc3 for HTTP and HTTPS taffic.

I can get to GRC.com! So changing nothing I reboot and NOW GRC.com is blocked so...time to brake it!

I go to the policy control uncheck content filter set to none and ok

go to content filter remove grc3

add a new profile name grc4 go to custom service tab enable Custom Service and Check Common Trusted/Forbidden List checked in Forbidden Web Sites I have *.grc*.com go back to policy control to add grc4 grc does not load uncheck content filter grc4 can get to grc.com re-enable can't get to grc.com....ok remove the policy control rule and add it back in grc blocked disable/Enable HTTPS Domain Filter for HTTPS traffic still blocked removed *.grc*.com test grc it loads add grc.com in Forbidden Web Sites grc loads...remove grc.com add *.grc*.com and YES grc loads so changing nothing reboot and... NOW HTTPS Domain Filter for HTTPS traffic works.

Zyxel_James

Hello @peterUK

According to your newer clip provided, I'm more sure it's a cache problem instead of an issue although it's not intuition that you need to clear the content filter cache before blocking.

In your clip, nordvpn.com was loaded first, so the device had the cache already. It's no doubt that you need to clear the cache before blocking it again.

James

PeterUK

Yes I get now the cache has to have a allow mainly for the content filter category or it be looking up all the time! so one quick fix would be to have the debug command run on a change to profile.

smb_corp_user

Sorry for nitpicking, please use the spelling "break" instead of "brake" when saying that something has stopped working. The word "brake" means to slow down, like using the brakes in your car or other vehicle. Use "break" to mean that something gets destroyed. (Thank you in advance.)

osake_li_09

I think it could be browser cache or content filter cache. You may use an incognito browser, and clear TCP sessions, CF cache before tests.

Router#debug conntrack flush    //clear TCP sessions

Router(config)#content filter cf-queue flush   //clear Content filter cache

PeterUK

Now that I'm the proud own of services with USG FLEX200 on V5.35 this is still a problem!

mMontana

New hashtags for firmwares: #keepingbugs.

Zyxel_James

Hello @peterUK

Since the paragraph has no punctuation, I'm not sure I understand right, please correct me if the steps are wrong. 

1. create a Content Filter profile "grc3", enable Custom Service and Check Common Trusted/Forbidden List, add *.grc*.com to Forbidden Web Sites, and apply to LAN1_outgoing >> grc.com is blocked

At this first step, you need to reboot the device to make the block work, I think it's a cache problem. You should use an incognito browser, and clear TCP sessions and CF cache before every tests.

Router#debug conntrack flush    //clear TCP sessions

Router(config)#content filter cf-queue flush   //clear Content filter cache
Moreover, you can clear dns cache by ipconfig/flushdns

2. Remove "grc3" from LAN1_outgoing and go to grc.com >> grc.com is loaded which is normal because CF profile is removed.

3. Create another CF profile "grc4" with the same settings and apply it to LAN1_outgoing >> grc.com is loaded. But actually, it's a browser cache problem. If you clear the browser cache and input the suggested command and ipconfig/flushdns, the page is blocked again.

4. The result is the same for the rest steps. If you clear browser cache and dns cache, the page is blocked again.

I suggest clearing the browser cache and DNS cache every time you test the blocked site, thank you.

James

PeterUK

Hi James thanks for your interest in this have tested again doing browser cache clearing there is still a issue.

So this really is broken if you put in grc.com for Forbidden instead of *grc*.com even with browser cached if you don't test this yourself you will never know how bad this bug is.

PeterUK

Ok update I have found why your not seeing it as its because your not testing with a bridge in my case WAN1 and DMZ are in a bridge the IP does not matter I use 127.255.255.125/30 PC on DMZ and internet on WAN1 as normal.

With a policy control for DMZ to WAN and the content filter rule

Then test with Forbidden Web Sites *grc*.com all is fine its blocked

Change Forbidden Web Sites to grc.com with browser cache cleared you can get to grc.com

Change Forbidden Web Sites back to *grc*.com with browser cache cleared you can still get to grc.com at which point you have to reboot the USG for content filter to work again.

Zyxel_James

@PeterUK, thanks for the details information.

I can reproduce the symptom with DMZ bridge and your steps. However, I will check on this and get back to you when there is any progress, thank you.

PeterUK

I have posted a this in Zyxel Support Campus under

#355158
One fix would be when you input like grc.com it auto enter it to *.grc*.com

Zyxel_James

Hello @peterUK

Except for the commands I suggested, please input these commands as well

debug content-filter https-domain-filter cache flush

debug system dmesg clean| match "grc"

I also suggest using an incognito browser for testing.

I reproduce the steps with input the commands before tests, and the website is blocked. It's a cache issue.