USG40's VPN VOIP Stopped Working

RDVasil
RDVasil Posts: 5
We have USG40's at a branch and main office connected to each other by IPSEC VPN.  Have VOIP phones at branch office and phone system controller at main office.  Worked fine for months but stopped working three days ago.  No changes made to either router, both running 4.63 firmware. 

VPN is working - computers on either side can see each other, ping and RDP work in both directions, computers at branch office can access file server at main office. 

The only problem is the phones - callers at branch office can make calls but cannot hear - no audio comes in.  Wireshark shows audio, which should be coming in on UDP ports 10020-10050, is not coming in, but on the main office side Wireshark shows audio is being transmitted.

My understanding is that all tunnel traffic should be enabled by default.  I've manually added policy to allow all traffic to/from the telephone controller at the main office to no avail.
Question - if I briefly turn off Policy Control on both routers, will that enable all traffic - and if the phones start working again, would that verify a policy issue?  Or, am I missing something else?
«1

Answers

  • Ohiobound
    Ohiobound Posts: 1
    RDVasil said:
    We have USG40's at a branch and main office connected to each other by IPSEC VPN.  Have VOIP phones at branch office and phone system controller at main office.  Worked fine for months but stopped working three days ago.  No changes made to either router, both running 4.63 firmware. 

    VPN is working - computers on either side can see each other, ping and RDP work in both directions, computers at branch office can access file server at main office. 

    The only problem is the phones - callers at branch office can make calls but cannot hear - no audio comes in.  Wireshark shows audio, which should be coming in on UDP ports 10020-10050, is not coming in, but on the main office side Wireshark shows audio is being transmitted.

    My understanding is that all tunnel traffic should be enabled by default.  I've manually added policy to allow all traffic to/from the telephone controller at the main office to no avail.
    Question - if I briefly turn off Policy Control on both routers, will that enable all traffic - and if the phones start working again, would that verify a policy issue?  Or, am I missing something else?
    Check the security incident alert email that came today.  There is probably a policy route created, a ssl vpn, and a fake user created.  I have had 7 firewalls with this issue.   Our voip quit working as well after this came about.

    Here's the email:

    Dear Customer,

    We recently became aware of a sophisticated threat actor targeting a small subset of Zyxel security appliances that have remote management or SSL VPN enabled, namely in the USG/ZyWALL, USG FLEX, ATP, and VPN series running on-premise ZLD firmware. Those running the Nebula cloud management mode are not affected.

    We’re aware of the situation and have been working our best to investigate and resolve it. The threat actor attempts to access a device through WAN; if successful, they then bypass authentication and establish SSL VPN tunnels with unknown user accounts, such as“zyxel_sllvpn”, “zyxel_ts”, or “zyxel_vpn_test”, to manipulate the device’s configuration.

    Based on our investigation so far, we believe maintaining a proper security policy for remote access is currently the most effective way to reduce the attack surface; therefore, we strongly recommend that you follow the guidance and the SOP below:


    1.

    Unless you must manage devices from the WAN side, disable HTTP/HTTPS services from WAN.

    2.

    If you still need to manage devices from the WAN side:

    enable Policy Control and add rules to only allow access from trusted source IP addresses; and

    enable GeoIP filtering to only allow access from trusted locations.


  • RDVasil
    RDVasil Posts: 5
    We're got IPSEC VPN configured, not SSL VPN.  I see no additional users created, nothing created under SSL VPN, and nothing but the default VPN policies.  Our VOIP problems began last weekend, just after the firmware upgrade
  • Zyxel_Jerry
    Zyxel_Jerry Posts: 495  Zyxel Employee
    Could you share your topology and the policy settings with us?
    What is your firmware running previously?
    If you have any concern to share the information in the post.
    You can private message the information to us.
  • RDVasil
    RDVasil Posts: 5
    Main office has static IP at 66.xxx.xxx.xxx, USG40 at 192.168.0.2, an NEC phone system at 192.168.0.241; Branch office has static IP at 66.xxx.xxx.xxx, USG40 at 192.168.2.254, domain controllers with DNS at 192.168.0.254 (main) and 192.168.2.252 (branch), both functioning correctly and accessible from either side, VOIP phones with DHCP reservations from 192.168.2.70-84; IPSEC Site to Site VPN between the two offices, created originally using Easy Mode Wizard, then deleted and recreated in Expert Mode.  Originally no policies other than the default, but we've tried adding some 'allow' policies to/from the phone system with no success.  Previous firmware was 4.32 (main) and 4.31 (branch), both now running 4.63.  Everything appears to work except the phones, which are not receiving audio across the VPN (UDP ports 10000 and above)
  • mMontana
    mMontana Posts: 420  Master Member
    edited June 26
    DHCP server used by phones is hosted by USG40 at branch office?
    VOIPBX is in main office?
    Does VOIPBX know the route from main office to branch office? (should be able to receive and respond packages from both 192.168.0.0/24 and 192.168.2.0/24
    In main and branch office, is LAN2 (which is not port 2, 3, 4 or 5 but a network zone) is off or into a different subnet than 192.168.2.0/24?
    Can you create, on main office, a firewall rule (security policy) for create alerts if some of the voip phones are connecting to the PBX (and back?)

  • RDVasil
    RDVasil Posts: 5
    edited June 26
    Yes, DHCP server used by phones is at branch and VOIPBX is at main.  There is handshaking between phones and PBX on TCP ports 5060/5080 that does take place - we can see it on traffic monitor and logs on both routers.  We can see phones at branch send UDP 3462 to PBX, and logs at main show it forwarded.  But PBX should reply back at UDP 10000+ and this does not reach the phones at branch.  We put wireshark between the PBX and router at main office and it does capture traffic (audio) at UDP 10000+, so PBX appears to be sending it
    LAN2, which is not in use, is set to 192.168.4.0/24 at both routers to avoid any conflict
    And again, this is not a new configuration.  It had been working without a problem for approximately 6 months 
  • mMontana
    mMontana Posts: 420  Master Member
    edited June 27
    @RDVasil sorry if my hints for trying to troubleshoot your issues are not focused as I would like to, only trying to help you find the reason of the problem.
    FWIW there are no policy routes for regulate traffic via IPSec, everyhing pass from and to tunnel.
    In any case, you can write a rule (with alert) to allow traffic 10000+ from in main site from lan to tunnel and from tunnel to lan in branch office. The rule should tell you "packages are coming/delivered as requested". Service should be right object to setup the port intervals.
    Also there's BWM to have a look at.
  • RDVasil
    RDVasil Posts: 5
    That's why it's confusing to me.  In the absence of any other policies, I thought all IPSEC traffic is by default allowed.  I've tried adding a policy (on both routers) just for UDP 10020-10531, added policies Any to Any with the Source or Destination the PBX, turned off Policy Control completely on both routers, and tried both ALG and BWM.  Strangely, with Log Alert, nothing shows up in the log reports or the traffic statistics for the UDP ports.  It seems like the high UDP port packets are simply being dropped, not controlled by policy.
  • Zyxel_Jerry
    Zyxel_Jerry Posts: 495  Zyxel Employee

    Please type the command on the Main office device. 
    Router> debug conntrack flush   
    This command help to clean the cache on the device.
    If the command does not work, please help to collect the packet on the Main office Lan site.
    We would like to check the packet when doing the phone call.
  • ChrisBys
    ChrisBys Posts: 4
    Hello I have an old VPN license (5 pcs) and I am wondering if it still is working against USG60W? Is it possible to uppgrade to an extender licence?
    /Chris

Security Highlight