MX65 - any way to gauge saturation?

[RumorConsumer]

Hello!

I have a network with an MX65. The location is a retreat center. I have about 15 APs around a 60 acre property. Its sparsely populated, maybe 25 people for about 80% of the time mostly doing texting (iMessage, Signal, WhatsApp) and some web browsing, Instagram, Youtube. No heavy web apps. No torrenting. The usage is pretty light over all. We have 100mbit fiber and it more than handles what we throw at it. 

Because we host events a couple times a month where the number of people onsite jumps up to about 120 for a weekend, is there a way I can look at the stats on the MX65 and tell how saturated it is in the same way I would check on a PC to see how much its paging out to determine if it needs more RAM?

Follow up question.. is it possible that my APs would start malfunctioning if the MX feels overloaded? I sometimes notice the 2.4ghz band seems to be underperforming and I can't figure out why. 5ghz seems fine. Could this be part of it? 

Thanks.

[NolanHerring]

With the MX you can look at the summary report and it will show you the CPU utilization. That is the only way I know of to gauge its performance from a hardware perspective.

2.4GHz is under performing because it is 2.4GHz and most likely dealing with a lot of other contenders and interferes in the 2.4GHz spectrum.

Run a microwave while trying to do a live stream on WhatsApp or Skype while on 2.4GHz, and you'll see it tank real quick. There are many other things that are 2.4GHz only from a consumer perspective, and they all have equal access to the spectrum.

Your only real course of action is to go 5GHz, which has much more spectrum and far less noise to deal with. You could make the SSID 5GHz only and that would take care of that. Or separate the SSID into two, one 5GHz only and the other 2.4GHz only

[RumorConsumer]

@NolanHerring  thanks so much for your response.

I live in an entirely clean RF environment. There isn't barely even cell signal out here, no competing networks and no microwaves. Does your warning about interference still apply? For reference, my Macbook Pro is currently reporting a noise ceiling (i think is the term) of -97dBm which is whisper quiet and allows for signals that would be drowned out in a city to reach pretty well here. 

[NolanHerring]

So if your environment is clean, then 2.4GHz will/should perform just fine.

Its limited to 20MHz, but that should still be plenty when your uplink is 100Mbps to the Internet.

That noise floor is good, so no issues there.

What specifically do you notice when connected to 2.4GHz that makes you think its slow. Are you running a speed test? Noticing packet loss? High retry rates?

[RumorConsumer]

Here is my summary for the last week. As you can see very low usage. We're on a 100/100 fiber line. I can't see where CPU utilization is. Help?

[NolanHerring]

Go to ORG>Summary Report

Choose the appliance and you'll see it there

[RumorConsumer]

Found it. I ran a report since April 1 (today is July 17 for anybody reading this in the future) and never go above 40% or on the Utilization score. Barely above 15% most of the time. So unless address space and # of clients period is an issue I am nowhere close to saturated or pushing this device at all. Do you all agree with that assessment? @SoCalRacer based on this image of my utilization would you still assert that Im over utilizing? 

@Nash a number are repeaters. The ones I have the most trouble with are actually the Gateways. 3 Gateways and about 10 repeaters. MR 84, 74s, and a few 42s and 52s mostly and the gateways are 74s and the 84. I don't have any of the rinkydink models. 

@NolanHerring I see literally a lack of 2.4 connectivity. 30 paces outdoors from a MR74 with omni's on the 2.4 connectors (top) that is 30 feet off the ground with clear line of sight using an iPhone X. In this environment. That should never happen. On the bottom connectors (5ghz) I have a directional 5ghz antenna. The phone does 5ghz til it dies at the 30 paces which is expected. I rebooted the AP last night and it might be better. Maybe just somehow forgot its 2.4ghz-ness. So strange. 

[Nash]

Your device utilization rate looks pretty even, and within an acceptable range imho.

I am assuming that there was a period of time where this worked the way you wanted it to, with regards to the 2.4 Ghz band.

If the rebooted AP is better now, is it possible to bounce the other problem child APs? I've had some weirdo stuff that sorted itself out after an AP reboot and never happened again. You might get lucky here.

[NolanHerring]

Just to confirm, your using/mixing antennas on your access points?

[SoCalRacer]

Somethings to clairfy:

1) You said you are using a MX64, but the screenshot shows MX65

2) Looks like your highest device usage was a TP-Link C2300 router, are you defining this as an AP?

3) You noted you have 120 clients on at a time, the highest date here shows less than 70

4) In 3 months you are under 50 clients with a couple days that peaked 70?

5) What is between the MX and the APs? (Switches?)

Personally I like to run this report daily or weekly. They analyze the trends. Then you can determine if something normal usage, trending, or an anomaly.

[NolanHerring]

Oh nevermind. I read MR84 but I see your using different antenna on the MR74. Forgot the MR74 has two 2.4 and two 5GHz connectors, so you can mix antenna types.

[RumorConsumer]

Well I screwed that up - it is an MX65 indeed not a 64. Sorry!

@Nash Re: A time when it worked

Yes thats true, and I have a feeling something got weird with the APs. I rebooted them all last night and so far this morning I still need to test out and see if they are better. Will report back shortly.

I power cycled the MX last night too for the first time in months so maybe that will help. 

@NolanHerring Man I was having collision central here and couldn't figure out why until I read the manual on the 84 and paired the antennas. Good lord. I wasn't on the set up team but when I finally realized we had a exposed node issue due to the dual band joined antennas in the 84. Big important detail!

@SoCalRacer 
1. You're right! I fixed it. 
2. That is being used on a ethernet line off an AP to give wireless to an isolated room that doesn't get very good mesh connectivity or signal penetration. Its not being counted as an AP.
3. Oops right again. I confused total unique with peak. 
4. That would be correct. Those higher points would be our events.
5. Yes, older Dell Powerconnect 3324 10/100 switches that also handle the fiber links around the property. Will be replacing them soon with current newer Netgear switches. This whole thread is a product of me seeing the next level up of Meraki MX devices do SFP which meant it could handle the fiber input and network connection. I thought ok why even bother getting a separate switch when I can just combine these two pieces of hardware. Make sense? 

[SoCalRacer]

I would say the MX could be undersized, but it wouldn't be my highest priority. My first priority would be to replace switches. Then ensure gigabit connections from MX to Switches to APs. I would also try to cleanup the topology behind the switches, like daisy chaining a C2300 router onto an AP. If possibly I would try to keep all APs in the same stack, then model if possible, unless use case determines a different need. Also a recommendation would be to use draw.io or something similar to document your network diagram. When it draw out and in front of you, things make a lot more sense and areas to address or common problems you have might be able to be easily diagnosed. Next I would track the 2 top highest client usages and see what kind of data they are using. Could be putting more pressure on the network. To me any one device that is using 15-25% of the total usage needs to be understood why. Maybe it is the highest use AP, but then maybe you can spread that usage across APs to help with speeds.

[RumorConsumer]

@SoCalRacer I just realized that C2300 is actually in my networking closet providing a cheap subnet for a NAS that does a lot of internet backup. Thats why its taking so much bandwidth. And its totally wired.

[SoCalRacer]

You might want to turn wifi off on the router if its just doing cheap subneting. Then maybe throttle back the backups during active hours. Looks like you are making progress already! 🍻

[BlakeRichardson]

The MX65 is only recommended for up to 50 clients, yes it will handle more but service quality will be degraded. To cover the large amount of people you have you would need an MX84.

The hard part will be working out if its a worthwhile upgrade or not since usually your client numbers are around 25.

[BlakeRichardson]

You could always turn Advanced Malware Protection off as well to reduce CPU usage but you would be doing so at your own risk. 

[RumorConsumer]

@BlakeRichardson thanks for the reply. Did you have a look at my CPU utilization screen shot I pasted in above? If there’s 70 clients who are doing relatively nothing and the CPU never spikes over 25% is that still true? What metric let’s us know the device is actually unable to maintain the load? What symptoms would I be seeing? I understand you’re saying the spec is 50 but with my kind of traffic does what you’re saying still hold true?

[Nash]

If you're looking at replacing your switches soon, since you've already got Meraki APs and firewall... This is a good time for Meraki switches if you can afford it. Then you'll have transparency through the entire network from ingress to egress.

Assuming all your APs are Meraki APs, ofc.

[RumorConsumer]

@Nash
Great thought and my sense is it’s out of our price range. Very nice hardware though and I’ll have Meraki on each side which will give me basics on what’s going on through the switches. Any other info about what a switch upgrade to modern ones will do? Our internet is again only 100/100 and we don’t really transfer files.

[Uberseehandel]

In my experience, 2.4 Ghz mesh/uplinking is asking for trouble. There are only 3 non-overlapping channels. With that, its tough to build a performing WiFi network.

If you have 120 people come in over a weekend, how do they live - are they in cabins, RVs? How do they cook? That many people in the area are significant as far as WiFi performance is concerned. Are they all tagged? Do they use mobile phones? Do they have microwaves, are their barbecues WiFi enabled?

[RumorConsumer]

@UberseehandelThat is great to know. Again, I am way way out there location wise so Im not sure if the non overlapping channel issue is an issue for me but I think its safe to say I am low man on the totem pole on this forum though I have been in IT for over 20 years. So Id listen to you guys. My main desire to use 2.4ghz for a mesh is a specific link over a long distance in the outskirts of the property. I have a few houses that are off the main locus of the property and getting wifi out there through the trees is hard. I am figuring out how i can hardline. 

The people who come are in existing structures. We have cabins, a few redwood houses. Everything very light constructions materials mostly built in the 1950s. No concrete. We have a dining hall where all food is made. Nobody cooks at home. No RVs or anything. Cars are all parked in a lot far away from all wifi hardware. These are a lot of San Francisco tech types so yeah phones and computers but mostly phones. We host weddings, retreats, etc.

[Uberseehandel]

"A lot of San Francisco tech types". Sounds more and more like the Bohemian Club.

Seriously, so many of those guys love turning their phones into Hot Spots. Which will be a source of interference.

"Mostly built in the 50's" - there is a possibility that some of the wiring is worn, and possibly there is neon lighting in the kitchen/foodhall. Neon lasts, but there can be interference problems especially if Ethernet cable is nearby.

You may feel it worth using a WiFi survey tool to see what happens at quiet and busy time. 

Heatmapper from Ekahau is a great free tool. Plenty of helpful information on their web site. People who have to survey a big area create some ingenious ways of lugging laptop and WiFi AP around to record the data. Indoors I use a shopping trolley. At home, we hooked a trailer onto a quadbike and I sat in there with laptop, power supply and AP as we surveyed the area to be covered. I'll expect some hilarious photographs.

[BlakeRichardson]

@RumorConsumer  Meraki's guide of 50 users would take into account 50 users using a MX that got a ton of its options enabled and being used.

Given you have 70 users and you are basically using it for internet and nothing else I wouldn't worry until people complain about speed issues or you upgrade your internet connection to more than 250Mbps.

[CptnCrnch]

Haven‘t tried it myself but could SNMP be solving the monitoring issue?

[SoCalRacer]

At your peak times I think the MX is under sized. Even at the low time with 25 users, I am assuming there are more than 50 devices behind the MX. Check the sizing guide below.

https://meraki.cisco.com/lib/pdf/meraki_whitepaper_mx_sizing_guide.pdf

@NolanHerring is spot on with the 2.4 and 5Ghz recommendations.

Also consider the AP you are using might be undersized or not getting a full gigabit link back to the MX. My thoughts would be that these needs to be MR33 or better APs. Check the link below to help you gauge usage.

https://documentation.meraki.com/Architectures_and_Best_Practices/Cisco_Meraki_Best_Practice_Design/...

[Nash]

Are any of these APs repeaters? Or are they all wired back to home base?

[Mike_LDS]

One other thing you might want to check is the wireless health report.  You can see from that report not only the failed connections but also the latency per AP.  That may show you where you are having wireless problems.

FYI, we have sites that routinely exceed the "suggested" number of clients with no issues.  It comes down to what type of traffic you are passing and the number of services you are running.  If it's simple internet traffic and your utilization isn't too high, then you should be ok on the MX.

One other thought on the Meraki reporting.  Keep in mind the longer your timespan, the more of a trending view you get.  When Meraki shows that report, they are not giving you an exact view of traffic.  They actually sample the data and show a summary.  So you may actually miss some peaks and valleys.  Once you understand that, then you can understand that the traffic reports give you a good "trend" but not exact data.