Intermittent loss of WAN connection - Meraki MX68W

[Matt7]

Morning. I replied to a similar thread, but perhaps best to start my own. Just looking for clues really...

We’re experiencing WAN dropouts. We have BTNet 100mpbs supplied to the Meraki MX68W as wires only, then into a Draytek 3910 for our GW firewall, DHCP.

Internet connection drops once a week, maybe 2 weeks, but on the Draytek the WAN shows as still live/connected. It will recovery by itself after a few minutes. But we’re using BT cloud voice, so users lose their calls for a few minutes.

I’m suspicious of the Meraki. We had an issue with it a few months back, where randomly one morning we could not get any internet access out from the Draytek at all, but BT could see the Draytek inbound, so stated the connection was fine. If I connected a laptop directly into another port on the Meraki it worked (is that wires only?!). So at a loss I changed the Draytek to that port on the Meraki and it resolved it! I made no other chages to anything on the network/router apart from changing to another port on the Meraki. BT refused to replace Meraki as it was now functioning.

I’m pretty certain it’s nothing inside the network, everything on the LAN side still works fine (Database/CRM, file servers etc). BT state no errors on the Meraki/line, but state that we are regularly using all of the bandwidth. I do see 95Mpbs usage reported on the Draytek at various times throughout the day (we have 60+ users). Is that a likely cause for connection instability/dropping temporarily?

I can upgrade bandwidth, but reluctant too spend if that's not the problem.

What doesn’t sit with me is the period of time between the drops, why only once a week or two.

[PhilipDAth]

If you plug into the MX and it works I'm less inclined to think it is the MX.

Is the firmware up to date on the Draytek?  It sounds like this could be a software bug.

[Matt7]

Yes firmware updated. I forgot to add, I've already switched to another router to rule it out.

Weird one!

[cmr]

@Matt7 Am I right in that you are using the MX and Draytek in line with both NATing?  May I ask why, is the MX managed by BT and can you not control it?

[Matt7]

That's right, the MX is managed by BT, and provided on a "wires only" basis. We have no access to it. We establish a WAN connection from the Draytek out to the MX. This works, but once a week or perhaps longer the connection drops routing, disconnects everyone from the internet, and then a few minutes later all comes back online without intervention.

[cmr]

You might be better with a layer 2 device like an ADVA, it doesn't sound like the MX is giving you any value and only potential incompatibilities.  Can you not at least get read only access to it, seems a strange choice for an ISP to issue a very capable edge device and neuter it...!

[Matt7]

The fibre does terminate at an ADVA box, we have one. Then ADVA into Meraki, then specific port on Meraki (I'm assuming BT configured that port as pass-through?) to Draytek. 

This does seem messy, but this how is BT specced and installed it. I wonder if I can simplify this..AVDA into Draytek

[cmr]

I think the MX may be routing, we usually have the fibre coming in, the full width 1U ADVA with the two power connectors on the left hand side, then a second 'smarter' half width ADVA with a single PSU that breaks out the services to individual ports. 

I'm guessing the MX is playing that role, but is way over engineered to be just splitting up a trunk into it's constituent VLANs.  For you if you only have pure Internet access it is pretty much just being a wire...

Before removing the MX do a traceroute to the Internet to see if you can tell which mode the MX is in and if in routed, what the IP on the MX WAN port is.  We had a similar problem with ISP kit in South Africa about a year ago and did remove the intermediate device and they never complained!

[Matt7]

Thanks for the responses. I agree it is over complicated.

Our ADVA is an "adva fsp 150-ge102pro". I've talked ot BT, it will work taking the MX out of the equation. But I need to place an order for them to change the config of our service.

They won't allow me to have any interaction with the Meraki, but I can plug directly into that without the Draytek. So one option I guess is to use the MX as my router - but then have no ability to manage that/see whats going on/monitor etc.

They state many customers have the same setup as we do without issue, ie. Draytek/customer's router > Meraki > ADVA.

I did compare traceroute using current setup vs plugging direclty into the MX. One thing I noticed was using my current setup (Drayek>Meraki>ADVA) it seems to double NAT, the first two entries are internal (this is beyond my knowledge!), when plugging straight into the MX just a single private IP.  I asked BT about this, she said the port I'm to use on the MX is configured 1 to 1 NAT. This is probably a non issue..

What still bugs me, is how random the issue is. If it was a basic config error it wouldn't route at all? but it's a week between internet drops. BT's only input was that sometimes we're using high bandwidth, so want to sell up 1Gbps

[cmr]

If they won't let you see what the MX does then ditch it, the whole point of Meraki is you get good visibility!  If they want you to go to 1Gb then BT would have to replace the MX68 as it only supports up to 450Mb.  What a waste of a good piece of equipment 🤦‍♂️