Community Record
104
Posts
54
Kudos
1
Solution
Badges
I disagree that clients on the same SSID will all see the same BC/MC traffic, regardless of the VLAN they’re assigned to. That’s broken. If I assign a client to a particular VLAN they should see link local multicast from only that VLAN. I decided to test iPSK as a substitute for group-policy assigned VLANs applied directly to clients. This feature works correctly. A client connecting with the PSK for VLAN 105 gets traffic from only VLAN 105. A client connecting with the PSK for VLAN 100 gets traffic only from VLAN 100. There’s no leakage of BC/MC traffic when using iPSK. Only when using client-specific group-policy assigned VLAN tags. I thought the multicast to unicast conversion feature might be contributing to this, but it didn’t change the behavior. The MR44 is still leaking link-local multicast across VLANs to clients with VLAN tags assigned via group policy. To confirm I didn’t have something wonky going on that I wasn’t aware of I created a completely new group policy that assigned VLAN 999. VLAN 999 doesn’t exist in my network, anywhere. It’s not trunked on any ports, and there are no devices in that VLAN. Theoretically a client assigned to VLAN 999 using a group policy shouldn’t see any traffic beyond what the client itself generates. In reality though the client still saw all the link local multicast traffic generated in the default VLAN assigned to the SSID. This feature is broken, and it’s especially bad from an IPv6 perspective given how much of the protocol is based on link-local multicast and ICMPv6. Leaking that beyond VLAN boundaries breaks a lot of things.
... View more
Dec 31 2024
7:29 PM
I did a little more investigation here. It would seem that the MR44 is leaking both ipv4 and ipv6 multicast packets sent on the base SSID access controlled VLAN (the default vlan), to clients in other VLANs if those VLANs are assigned via group policy on the client. Specifically it seems to be leaking the 224.0.0.0 multicast range on IPv4 and the ff02:: range on IPV6. This is especially problematic because leaking the IPV6 ff02:: range causes the client with the group policy attached to received ICMPv6 messages from both its assigned VLAN and also the default ssid VLAN. So, that means the client is getting ICMPv6 neighbor solicitation requests from devices it can't actually respond to. I wonder if it forwards Router Advertisements also, I bet it does.
... View more
Dec 31 2024
5:30 PM
Is anyone here using group policies to set different VLAN tags, per client, on the same SSID? For example, I have the following: ssid: foo ssid access control vlan tag configured: 105 mode: bridged auth: basic psk wpa2 vlan 105 network: 192.168.5.0/24 vlan 100 network: 192.168.1.0/24 Scenario 1: Clients without a group policy attached can connect to the "foo" SSID and get an IP address in subnet 192.168.5.0/24 from the DHCP server on VLAN 105. mDNS traffic is seen from the 192.168.5.0/24 subnet. This works as expected. Scenario 2: A new client connects to the "foo" ssid. This particular client has a group policy attached that sets the VLAN tag to 100. No other options are configured on the group policy. The client gets an IP address in the 192.168.1.0/24 subnet from the VLAN 100 DHCP server. mDNS traffic is seen from the 192.168.1.0/24 subnet AND the 192.168.5.0/24 subnet. That's wrong, the client exists in VLAN 100, not VLAN 105, and should not see mDNS broadcasts from devices on VLAN105. This can easily be seen using an mDNS discovery tool (Discovery.app from the Mac OS App Store for example) or a Wireshark capture on udp port 5353. Has anyone else encountered this? This makes group policy assigned VLAN tags completely useless if traffic leaks between VLANs at the AP. Note, there's no bonjour forwarding / mdns reflectors in play here that could explain this traffic flow. If I build a new SSID that hooks directly to VLAN 100 via the SSID access control settings there are no mDNS broadcasts from VLAN 105 seen on the client device, as expected. I'm running MR44s on MR 31.1.5.1 and MS120s on 17.1.3. If anyone has any ideas that would be great.
... View more
Labels:
- Labels:
-
Other
Oct 24 2021
2:35 PM
5 Kudos
This appears to have been resolved as of 16.13. My MX85 is performing at least on par with expectations at this point.
... View more
Sep 22 2021
6:13 PM
For the sake of thoroughness I ran some additional numbers on the passthrough test case. Test client: 2018 Macbook Pro, MacOS 11.6, Safari 15.0. I am testing using fast.com My fast.com settings are: Connections: 8 min, 16 max Duration: 15 sec min, 30 sec max Test Case 1: test client -- MX85 (passthrough) -- MS120-8LP -- MS120-8 -- MS120-8LP -- Cisco Firepower 1010 ASA Mode 9.16(2)3 -- Google Fiber -- Fast.com Results - download / upload: 630/770 390/780 640/790 620/790 560/780 510/740 520/730 680/710 All numbers in mbps. Test Case 2: test client -- RJ45 coupler -- MS120-8LP -- MS120-8 -- MS120-8LP -- Cisco Firepower 1010 ASA Mode 9.16(2)3 -- Google Fiber -- Fast.com Results - download / upload: 1.1gbps/710 1.1gbps/740 990/770 1.0gbps/730 1.0gbps/790 1.0gbps/730 990/720 1.0gbps/750 All numbers in mbps unless otherwise indicated. Interestingly, upload speeds seem to be fairly consistent. Download speeds are not. I've tested this on 16.8 and 16.12. The results are the same. No other clients are connected to the MX85 except this test client.
... View more
Sep 21 2021
12:06 PM
1 Kudo
@NateF I do not have any of those features enabled. The MX is attached a completely new MX network, the config is default. There's no firewall rules, no IPS, no AMP, nothing. Just raw L3/L4 firewalling.
... View more
Sep 20 2021
7:29 PM
I'm going to reply to my own thread again just to keep documentation of the testing I'm doing. With the MX85 out of as my main router, I've now got this topology: Google Fiber (1gbps) --- FP1010 (asa mode version 9.16(2)3) --- various MS120-8 switches --- MX85 (bridge mode) --- test client In this setup, the MX85 is in bridge mode. The rest of the MX network settings are default. I'm doing a simple speedtest using fast.com. The results are all over the map. Some as low as 250mbps down, to as high as 650-700mbps. Nothing that shows a 1gbps. If I disconnect both cables from the MX85 and connect them together using an RJ45 coupler (eww), I immediately get 1gbps download on the next test on fast.com. I'm not sure what else to even look at. I'll get to a point where I think I've narrowed down a set of consistent test results / conditions only to have the results change dramatically on the next test. Another oddity is that I've been getting a warning in the dashboard that the MX isn't connected using TCP/443 for its tunnel to Meraki when attempting a code upgrade (16.8 -> 16.12). I never saw that on the MX67, ever. There is no firewall blocking it. Maybe there's a process on the MX85 that keeps crashing / restarting causing unnecessary processor load that would explain the wonky performance.
... View more
Sep 20 2021
6:17 PM
2 Kudos
@ww Are you using VLANs in your configuration? I've adjusted my test setup and moved my MX85 up to my test bench. I also moved it to a fresh MX network so it has a near default configuration on it. My results are: No VLANs - test client connected to port 5 - 960mbps combined upload/download throughput. So this is pretty much near line rate, which is the expected performance. 5 VLANs (1, 10, 20, 30, 40) - test client connected to port 5, access port, VLAN 1, consistently gets a combined throughput of ~860ish mbps. The behavior is repeatable. The weird thing about it, the extra VLANs, 10, 20, 30, 40, don't have any active clients, or, any active ports. They're simply configured. VLAN1 is the only VLAN that's actually in use because that's the only client connected. In either test scenario the test client is just an access port in VLAN1. It's unclear to me why just the presence of other VLANs (everything else is MX network defaults) would drop the performance so severely. The MX67 this MX85 is going to replace had no issues hitting the 450mbps rated max, even with VLANs enabled.
... View more
Sep 20 2021
6:53 AM
@cmr You are correct, MX 16 seems to be the only supported version at the moment. I obviously recognize that it's the beta release, but I don't have any other options. This performance delta is not called out as a known issue. If it is, it should be listed.
... View more
Sep 20 2021
6:49 AM
1 Kudo
@Dunky It's not supported on v14. v16 the only supported train at the moment.
... View more
Sep 19 2021
4:43 PM
1 Kudo
Following up on my own post here. I replaced the MX85 with a Cisco Firepower1010 running ASA 9.16(2)3. It performed both tests, Test1 and Test2 at full line rate (930mbps). This is a direct swap using the same cables, same test hosts, same WAN link, same background noise traffic. The MX85 is not performing to spec on 16.12.
... View more
Sep 16 2021
5:07 PM
@Brash While I certainly agree that routed operation takes more horsepower than switching, this is a question of the device performing to the specs as outlined in the datasheet, which it currently is not, by almost 25%. These are large (1500 byte) packets, in a single long lived flow. This should be one of the easiest flows to reach the max throughput of this device. I will note that there is other background traffic/noise present on the network but not enough to explain the consistent impact to the performance. That noise would also impact Test1, which is consistently reporting line rate transfers on the gigabit links. The results thus far are very consistent which rules out any impact from the background traffic. Interestingly, when I disabled traffic analytics I saw a small, but measurable difference in throughput, 730 vs 700. My theory was that disabling traffic analytics, and thus NBAR2, would lower the CPU-load on the device. My results suggest that this is the case. This would 100% reproducible as well. I've got a case open with Meraki support but I was curious to know what the rest of the community is seeing.
... View more
Sep 16 2021
2:13 PM
Has anyone else tested an MX85 and actually reached the stated 1gbps performance through it? I received mine yesterday, it's running 16.11, and I can only get 700mbps through it. I'm testing internal-to-internal only flows, VLAN to VLAN, to rule out issues with my WAN. Topology: Client --- (port 5) MX85 (port 6) --- Server Client - Mac Mini 2018, Mac OS 11.6. Server - Ubuntu 20.04.2 virtual machine, 4 vCPU, 4gigs RAM Test - Copy 8gig file using scp from the client to the server. Test 1 - same VLAN. Topology: client --- VLAN 10 --- server In this test, the copy runs are wire rate, 1gbps. Using Dashboard I see a reported rate of approximately 930mbps for the switchport the client is connected to. This is the expected result as the MX is not involved in this flow beyond simply switching the traffic. Test 2 - different VLAN. Topology: client --- VLAN 10 --- MX85 --- VLAN 20 --- server In this test the server is moved to a different VLAN, VLAN 20. In this test the copy runs at approximately 700mbps. Using Dashboard I see a reported rate of approximately 700mbps for the switchport the client is connected to. The only difference in this test is that the traffic must be routed by the MX. All physical cabling is the same between the two tests. To move the server I just change it's VLAN assignment. Because this is VLAN to VLAN, internal-only, traffic there is no NAT. The traffic-shaping configuration doesn't apply because the WAN port isn't being used. The firewall rule permitting this traffic is the first one in the list, so it shouldn't be a rule lookup issue. I thought maybe this was a per-flow limitation, so I brought up a second client and ran the test concurrently with the original client. What happens there is the performance is cut in half, with each client only getting about 350mbps through the MX. Note, I only have the Enterprise license so there's no threat protection (snort, amp, threatgrid, etc) enabled. The datasheet number of stateful firewall speeds of 1gpbs should apply here as that's all I'm asking the MX to do.
... View more
May 23 2020
6:05 PM
Unfortunately, bug/defects like this are resolved in priority order. A bug that's 24 months old that is affecting one person will take a back seat to one that's brand new but is affecting dozens. That's just how it works, good, bad, or whatever. It works like this everywhere.
... View more
May 23 2020
12:37 PM
1 Kudo
I’m sad to say that a fix for this is probably not going to arrive any time soon. The amount of people doing DHCP relay to an MX has to be very, very low. These devices are targeted at a very specific customer-set and people doing DHCP relay to the MX itself are not likely to be very common. If this issue is a show-stopper in your environment just toss a dhcp-pool on the switch that’s doing you’re DHCP helper and call it a day.
... View more
May 20 2020
9:31 AM
I'm sorry, but I don't understand this statement: "Seeing it on my syslog server without any way to determine which rule it came from is not acceptable to us or our clients." It says, in the syslog message, which pattern it came from. Sure, it doesn't say "Rule 15", but it does give you enough information to determine what rule it is from. It's not ideal, but the information is available to you.
... View more
May 20 2020
7:35 AM
1 Kudo
Well, you could make that argument about anything it does. There's no way for you to get into the MX to see the actual ruleset that's configured. If that's a requirement for your environment you may want to consider a different product/solution that offers you that level of control / visibility. Regarding your rules not generating hits. Are you saying in the Dashboard? If so, I've found those counters to not be real-time. The logging option on the rule is your best bet. However, I assume that is probably rate-limited to avoid causing a DoS on your own equipment by trying to log too much info too fast. Even regular big Cisco IOS/IOS-XE routers have a logging rate and queue limit.
... View more
May 20 2020
7:21 AM
Agreed, it doesn't tell you "rule 25 generated this log". You're going to have to determine that from the rest of the message contents. For example: 1589984219.325305969 mx67 flows src=192.168.1.82 dst=192.168.40.254 mac=AA:BB:CC:DD:EE:FF protocol=tcp sport=46928 dport=853 pattern: deny tcp && (dst port 53 || dst port 853) Specifically, you need to look at the pattern. This will match what you've configured in your rules. The one from the above example is a Deny rule that blocks all TCP traffic to destination ports 53 and 853. Presumably you don't have several duplicate rules with the same matching criteria.
... View more
May 20 2020
6:59 AM
3 Kudos
The only way to see what firewall rules are getting hit is to configure a syslog server (Network Wide ---> General) and turn on the "flows" option. After that check the syslog checkbox in the firewall rule. Sadly, there's no other way to do it. To figure out which rule is generating the syslog message you need to look at the pattern show in the syslog message itself, it should match whatever you configured on the rule.
... View more
I had the same problem with my HP wireless printer on 26.6.1. It was an ARP related issue which was corrected in 26.7.
... View more
Jun 26 2019
1:40 PM
I tried the 0.0.0.0/0 and it works. I tested it by creating a rule and setting an abitrary DSCP value and then running a pcap on the outside interface to confirm that DSCP value was being set. Thanks!
... View more
Jun 26 2019
11:25 AM
I'd like to create a traffic shaping rule on my MX67 that matches all traffic not matched by the higher priority rules. Is there a particular syntax to match everything? I can't seem to figure it out.
... View more
Jun 14 2019
5:17 PM
4 Kudos
Group policy rules are not stateful. Only the firewall configuration page (Security & SD Wan --> Configured --> Firewall) is stateful rules. Group policy rules are basically ACL entries with no state, if you're used to configuring Cisco routers. You'll need to manually allow return traffic if you're planning to use group policy rules. Note, any rules you apply via group policy will bypass any rules you've got specified in the firewall configuration page. So, if you're using group policy to control internal vlan to vlan traffic, but also want the firewall configuration to apply to vlan to wan traffic you're out of luck. It's a disappointing limitation which has dramatically reduced the usefulness of group policy for me.
... View more
May 3 2019
6:51 AM
What is everyone doing from a combined/separate network perspective? I just started out with a full Meraki stack, MX, MS, and MR, so I started with a combined network. However, it seems that within a combined network I only have one chance to apply a group policy. For example, I apply a group policy to a wireless client device to block access to a particular internal resource via a L3 firewall rule I also have to completely duplicate all the L3 firewall rules that are configured on the MX, otherwise the device bypasses all the MX firewall rules. So, now I've split the networks. This seems to let me apply a group policy to my wireless clients (say, IoT devices), but also allows the MX L3 firewall rules to still apply because the group policies are only network specific. Is my understanding correct?
... View more
My Accepted Solutions
| Subject | Views | Posted |
|---|---|---|
| 11555 | Jun 14 2019 5:17 PM |
My Top Kudoed Posts
| Subject | Kudos | Views |
|---|---|---|
| 5 | 7839 | |
| 4 | 11555 | |
| 3 | 9707 | |
| 2 | 411 | |
| 2 | 8313 |
