MX67 WAN Speed dropping

[Unexus]

MX67 on latest stable version MX 18.211.5.1

We have a new ISP with 1Gbps link

We connect CAT6A to the ISP CPE device

Using static IPV4 provided by ISP on WAN1 P3 connected a laptop on VLAN1 192.168.1.0/24 DHCP

When nothing else connected, just this laptop I do a speedtest provided by the ISP just after a cold boot (power disconnect/reconnect). This speedtest shows approx 700Mbps, the limit of the MX67. So far so good.

Running another speedtest random after the first one I only get values around 200Mpbs. What is cauasing this? The WAN port shows hovering over 1GBps Auto Negotiate (as the ISP also tells us to configure our WAN port). The LAN port P3 also show 1Gbps Auto Negotiate.

When I program the laptop NIC with another static IP the ISP provides in a subnet an connect it directly to the CPE i constantly run approx 900-1000Mbps with speedtests.

So my conclusion the MX67 is causing an issue on the WAN speed.

We also run 2 site-to-site VPNs over this WAN connection. Can this cause the issue of dropping speed?

[RWelch]

Not all speed test sites are created equal.

Running a second or third speed test might have significantly different results from the first speed test based on where the 2nd/3rd speed tests (endpoint) are located - or how far away.  A way to tell if you are on a decent test site is to look at the latency/jitter column (results), lower is better.  Another quick way to see if the site you are testing on is by looking at the upload and download chart (graph)....if the upload/download lines all over the place - you might want to choose another speed test site.  You might have to be a bit selective in choosing which test sites to use by going with some of the more mainstream providers versus some of these smaller test sites or less familiar named sites.

Do you have any group policies or traffic shaping (per client bandwidth limit) in the equation?

The two S2S VPNs on this WAN could very well impact your speed test results if other users' data is coming in or going out to those sites - yes, it could impact your speed test results.  If there are other users/devices on the MX67 their network activity could also impact your speed test results.

[Unexus]

The speedtest I run is provided by my ISP. When I run this one directly on the WAN connection (MX67 replaced by a laptop) I measure around 1Gbps constantly. When the laptop is replaced by the MX67 I measure once 700Mbps, then constantly around 200Mbps.

No Group policies and/or traffic shaping per client enabled. The last one is set to unlimited.

I will investigate the VPN's whether they are influencing the speed

I see Cisco mentions a Maximum VPN Throughput of 400Mbps, so what does this mean for the WAN interface, will it drop to a lower rate due to VPN's that also run over it? Do I have to separete WAN and VPN traffic, e.g. VPN traffic over WAN2 and plain internet over WAN1? 

[RWelch]

The ability to run 1Gig throughput when testing without your MX67 would be the full ISP bandwidth.

The ability to run 700mbps throughput when testing with the MX67 would be limited based on the MX model you are using.

A subsequent 2nd or 3rd speed test to the same ISP could be due to either browser or app cached data or your ISP might not truly be evaluating a new inbound thread after just re-running the same test.  It could be it's being evaluated as a concurrent session since you re-ran the same test moments later or even minutes later.

A direct MX67 test to your ISP would (should) yield the capacity of the MX model.  Having turned on NMVPN to two sites would be additional factors that your WAN is working to provide you service with/to.

The 400mbps VPN limitations would be what you'd expect (best case) when sending traffic to/from S2S or NMVPNs.

Perhaps testing your speed with a different test site than your ISP would give you more consistent results. Or it might give you a better indication.  Bear in mind that some of the larger and more reliable ISPs that offer speed tests are truly able to handle multiple users/devices, higher overhead bandwidth capacity of running multiple and simultaneous speed tests concurrently - whereas your ISP test machine (site or server) might not be able to handle that same scenario.

Not sure where you are located but I tend to run speed tests against 3 different Google speed test locations and sometimes to get a 4th result will use Comcast.  

[cmr]

Are the S2S VPNs Meraki or other?  If Meraki you can see how much bandwidth they are using on the Security & SD-WAN/VPN Status page.

[Unexus]

Both Non-Meraki

[RWelch]

When you setup DHCP scope for client devices, how are you configuring the DNS nameserver (for DHCP responses)?

I tend to specify nameservers to those which provide the lowest latency and that tends to help client devices have better service/performance.  In the USA, CloudFlare and Google offer the lowest latency and I tend to use them for that reason.

Not familiar with how you setup client DHCP scopes but might be worth looking at how yours is setup/configured.

Configuring DNS Nameservers for DHCP and this would be a separate setting/configuration from the your other post about the WAN DNS not saved (WAN Uplink).

[PhilipDAth]

Start by making sure you are running a stable or better firmware release.

It is common for AMP/IPS to suck down the performance of the MX as it starts tracking more and more data.  You could you try turning off these two (under Threat protection) and see what impact that has.

If it does have an impact, you could try changing the ruleset to one with less impact on the CPU.