The team ordered a modem from one of the eBay links Meraki support provided. When it arrived, we noticed it was marked "engineering sample - not for resale". But when connected to the MX, it was recognized. So, we tried to activate service with Verizon and the sales rep could not do it. He got an error "ESN IS NOT E911/NETWORK COMPLIANT". Now this only happened because all of the Verizon modems on the Meraki compatibility list are EOL and can't be purchased new anywhere. I would have rather bought new from a reliable vendor. I keep having to tell myself that Meraki is a major brand owned by Cisco and that I didn't buy firewalls from some guy building them from open source software in his basement. It is an embarrassment that the product team is content with being equivalent to that.
... View more
It looks like Source Based Default Routing (SBDR) in the 15.4 firmware (which is stable release candidate now) may work for some use case scenarios. It looks like it supports routing all of a VLAN 's traffic across the VPN. If you have VOIP phones on their own VLAN and want to send all of the traffic through to a central gateway before hitting the internet, SBDR could work.
... View more
It's my general preference to prefer external modems. In theory, that makes it easier to upgrade the modem in the future. I have to operate as if the Meraki list is accurate. It sounds like you might be thinking the list is out-of-date and that true modem support is really much larger. But I don't have any confidence that is the situation. All the evidence I've seen leads me to believe that support is as poor as the list indicates. That the USB ID needs to match means there's more going on than just Meraki officially testing the equipment. It indicates there is special code that activates only when the applicable USB IDs are detected, and the real problem is likely lack of engineering resources to write the code for the newer devices.
... View more
Some of the devices I'm looking to add 4G failover to do have both Ethernet connections in use (same carrier for both circuits), and others don't. But I would like to set up cellular-specific rules and the MX has to know the circuit is cellular (e.g. be a supported USB modem) for that to be an option. I'm guessing you're using the Verizon 730L with older MX devices as the compatibility list indicates support was dropped for MX deviecs released after 2017.
... View more
I'm trying to set up cellular fail-over on the Meraki MX devices and I can't believe the ***tshow that this has become. Meraki's documentation is very clear - in order to use the cellular fail-over, a USB modem from the certified list must be used. https://documentation.meraki.com/MX/Cellular/3G%2F%2F4G_Cellular_Failover_with_USB_Modems My suppliers have told me that all of the certified modems are EOL. Skydus DS is one of the modems on the list and Inseego had this comment: "The Skyus DS platform is now EOL. ...the newer device is the Skyus DS2.... If connecting with a Meraki appliance, the DS2 will not work. Meraki has not / will not certify us on their platform any longer." I opened a support case with Meraki and got the response that "Engineering is already working on adding more devices to the list of compatible USB modems." Timeframe? - Nope. The support engineer, wanting to do something to help me, found some of the old modems on eBay. Points to the rep for wanting to help, but that Meraki engineering and product management let it get to this point is disgraceful.
... View more
I've tweaked the rule and re-established the desired routing behavior. As before, this is not a supported feature and Meraki can break it at any time with an update. Of course, the hope is that they release an official method to accomplish this, as this is a basic feature that is very useful to have on an "SD-WAN" device. The modified technique is to leave the rule enabled and then switch the condition to "While host responds to ping". Because the rule would create a routing loop, the host would never be able to respond to ping if the rule were active. The MX still advertises the route to its peers, but then sends the traffic out the internet when it arrives.
... View more
It seems that day had finally come. On Friday, I found the work-around not working and the desired traffic going direct to the internet from each spoke site. I'm not sure how long it had been like this. I had expected a firmware update would be the cause of this, but the one is installed is the 14.53 release from 6/2/2020. It's possible Meraki released an update to the cloud controller, and that changes how the rules are downloaded to the devices.
... View more
I'm not asking for them to add any features. I'm just asking them to fix a bug in a feature they have already chosen to deliver. It's not unreasonable to want them to do that in a timely manner. It wouldn't be correct to add the rules to the template because the port forwards are different at each site.
... View more
I have several networks set up using templates. And some sites have needs for port forwarding from the public IP address. Rules are configured in Security & SD-WAN > Firewall in the Forwarding rules overrides section. Example: This works once when the rules are initially added. If I try to make any changes at all, though, saving fails with a generic "Failed to Save" error. If I want to change the description of a rule, save fails. If I want to add a rule, save fails. If I accidentally left the protocol at UDP and try to change it to TCP, save fails. I opened a support case Feb 17 2020. To get me through the immediate need, the work-around was eventually discovered that I need to delete all of the port forwarding rules and re-enter them all by hand. If I make a mistake during re-entry, I need to restart and delete everything again. That was acceptable for a quick fix, but it's been almost a year now and the bug has not been fixed. There is no indication that the issue is limited to me. It probably impacts other customers that use templates and port forward overrides. While I don't know the full details, I am a software developer myself and this seems like it could be fixed by a junior developer within a few hours. When I ask for an update of the case status, I'm told no update exists. If I request an escalation, the request is denied. I asked my VAR for assistance and he told me he's not getting responses. (Oddly enough, he got a response when I asked for a quote to buy an additional license.) This is not the kind of support experience I expected from an enterprise network vendor. I expected that bug reports would be triaged and worked into the development schedule. There is a difference between prioritizing something and completely ignoring it. I read a Veeam post about how their support system has an automatic escalation mechanism if a case has been open for too long. With Meraki, there isn't even a way to manually escalate a case.
... View more
Meraki DHCP is set to provide the domain suffix (option 15). But none of the Ricoh printers are using DHCP. I also noticed that some of the printers that were successfully scanning to email didn't have the correct value in the field. It seems just being blank was a problem.
... View more
It appears that the problem has been resolved. All of the Ricoh printers are configured with static IPs in their settings. In Configuration > IPv4 settings, there is an option for Domain Name. The Ricohs that weren't working either had it set to DHCP or had it set to Specfiy with a blank value. After I entered a value and saved, scan to email started working. Some of these printers had been set for this like years without problems. Part of me still doesn't believe this fixed things (because why would it?) But it's been several days and none of the users have reported problems since they were informed scan to email was working. Thanks for all the suggestions and trying to help me resolve this. I appreciate it. This was a great experience.
... View more
Not that I can tell. And it gets weirder. Templates are being used as much as possible. There are 3 sites on one template communicating that scan to email is not working, and 1 on the same template that is scanning fine every day.
... View more
I set AMP and IDP to disabled on both the hub and spoke MX, and the scan to email still failed with "Proxy layer started discarding data. Acking message as failed." on the Exchange server.
... View more
I'm part of a team involved in a WAN upgrade project. We've been replacing EOL Cisco routers with Meraki MX67s. Auto-VPN is used to connect the sites to a central hub. There is an MX100 at the hub. At multiple locations, immediately after installing the MX67, Ricoh scan to email started to fail. However, at some other locations, it continued to work. Technicians have done an A:B comparison of the Ricoh settings between the sites that work and the sites that don't and haven't found any obvious differences. The Ricoh scan settings are configured to use an internal email server located at the hub site connecting via plain non-SSL SMTP over port 25. The data only has to travel over the site-to-site VPN tunnel. It never touches the internet. The Ricoh logs have generic errors that you'll find years worth of posts about in an internet search. Many of those have to do with authentication problems with Office 365 or Google G-Suite. That's not the case here. The email server is happy to accept unencrypted, unauthenticated communication. Here is an example from the Ricoh log: #[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (501) ERR:
#[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (701) ERR:
#[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (801) ERR: In the Exchange SmtpReceive log, we'll see something like the following: 2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,0,10.10.1.103:25,192.168.2.99:61118,+,,
2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,1,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,2,10.10.1.103:25,192.168.2.99:61118,>,"220 EXCHANGE1.domain.local Microsoft ESMTP MAIL Service ready at Mon, 21 Sep 2020 10:23:21 -0400",
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,3,10.10.1.103:25,192.168.2.99:61118,<,HELO RNP5838793490DB,
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,4,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,5,10.10.1.103:25,192.168.2.99:61118,>,250 EXCHANGE1.domain.local Hello [192.168.2.99],
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,6,10.10.1.103:25,192.168.2.99:61118,<,MAIL FROM:<email@example.com>,
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,7,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,8,10.10.1.103:25,192.168.2.99:61118,*,08D85DF4C6DA20F2;2020-09-21T14:23:21.484Z;1,receiving message
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,9,10.10.1.103:25,192.168.2.99:61118,>,250 2.1.0 Sender OK,
2020-09-21T14:23:21.625Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,10,10.10.1.103:25,192.168.2.99:61118,<,RCPT TO:<firstname.lastname@example.org>,
2020-09-21T14:23:21.625Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,11,10.10.1.103:25,192.168.2.99:61118,>,250 2.1.5 Recipient OK,
2020-09-21T14:23:21.671Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,12,10.10.1.103:25,192.168.2.99:61118,<,DATA,
2020-09-21T14:23:21.671Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,13,10.10.1.103:25,192.168.2.99:61118,>,354 Start mail input; end with <CRLF>.<CRLF>,
2020-09-21T14:23:21.718Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,14,10.10.1.103:25,192.168.2.99:61118,*,,Proxy destination(s) obtained from OnProxyInboundMessage event
2020-09-21T14:23:21.875Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,15,10.10.1.103:25,192.168.2.99:61118,-,,Remote(ConnectionReset) We see that communication starts, the recipient and sender are accepted, and then the connection resets after the Ricoh device is prompted for the data. In the Exchange SmtpSend log, the following is found: 2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,223,10.10.1.103:16585,10.10.1.104:2525,>,XPROXYFROM SID=08D85DF4C6DA20F2 IP=192.168.2.99 PORT=61118 DOMAIN=RNP5838793490DB SEQNUM=1 PERMS=1073 AUTHsrc=Anonymous,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,224,10.10.1.103:16585,10.10.1.104:2525,<,250 XProxyFrom accepted,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,225,10.10.1.103:16585,10.10.1.104:2525,*,,sending message with RecordId 0 and InternetMessageId <20200921093103GH.DCSML-S000010000.5838793490DB@192.168.2.99>
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,226,10.10.1.103:16585,10.10.1.104:2525,>,MAIL FROM:<email@example.com> SIZE=0 AUTH=<>,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,227,10.10.1.103:16585,10.10.1.104:2525,>,RCPT TO:<firstname.lastname@example.org>,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,228,10.10.1.103:16585,10.10.1.104:2525,<,250 2.1.0 Sender OK,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,229,10.10.1.103:16585,10.10.1.104:2525,<,250 2.1.5 Recipient OK,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,230,10.10.1.103:16585,10.10.1.104:2525,>,DATA,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,231,10.10.1.103:16585,10.10.1.104:2525,<,354 Start mail input; end with <CRLF>.<CRLF>,
2020-09-21T14:23:21.875Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,232,10.10.1.103:16585,10.10.1.104:2525,*,,Proxy layer started discarding data. Acking message as failed.
2020-09-21T14:23:21.875Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,233,10.10.1.103:16585,10.10.1.104:2525,-,,Local
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,122,10.10.1.103:16982,10.10.1.103:2525,*,,Proxying inbound session with session id 08D85DF4C6DA20F4
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,123,10.10.1.103:16982,10.10.1.103:2525,>,RSET,
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,124,10.10.1.103:16982,10.10.1.103:2525,<,250 2.0.0 Resetting, The line of interest there is "Proxy layer started discarding data. Acking message as failed." So the Exchange server is failing the message after it receives what it considers junk. Since this is all internal traffic, Meraki Advanced Malware Protection (AMP) shouldn't be involved, but could one of the Merakis be altering the traffic? I found a suggestion online to disable "SMTP inspection" if using a Cisco ASA. Obviously, nothing like that is present in the Meraki cloud controller, but is there a similar feature running on a Meraki MX? I did try opening a support case with Meraki, and the support rep did packet logging while an end user ran some scans. The technician didn't see anything of note. But the fact is that at some locations before Meraki MX67, scan to email worked. And after it didn't. Any suggestions you might have would be greatly appreciated.
... View more
As of March 2021, this method does not work. See my 3/7/2021 post for a revised method. The solution I had posted in November 2019 has been working. The short of it is that you create a static route on the hub MX for the internet addresses you'd like to through the VPN tunnel. For next hop, you put the hub MX IP. This would create a routing loop. So, you want to uncheck the Enabled box. This will cause the route to be advertised through the VPN tunnel by the hub MX. The spoke MX will see the route advertised by the hub MX and send the traffic through the VPN tunnel. Then when it arrives, the hub MX will send it out the WAN interface. One would expect the hub MX to stop advertising routes that have been disabled, but that hasn't happened in the 3 months since I set this up. I opened a support case to get clarification, since the setup seems a bit suspect. The support rep communicated with the development team and relayed the following: 1. This is not intended behavior. While the team doesn't have any plans to actively change how it is working, a future fix for an unrelated issue could have the side effect of closing up the loophole. So, it's use at your own risk. 2. The team does plan to implement a supported way to accomplish selective routing of internet traffic through VPN tunnels. My plan is to ride out the work-around until the supported solution is available. The worst that can happen is that the work-around stops working before the official solution is available and I'm back to where I started. But right now things are working exactly as I wanted, and I didn't have to buy any extra equipment. If you are doing pre-purchase research, don't base your decision on this work-around, because it is unsupported and can disappear. But if you've already got Meraki gear running, you don't have much to lose by trying it out.
... View more
This has been working great. I used this technique to configure VOIP traffic to go through the VPN tunnel out a reliable data center fiber connection. A colleague and I tested a phone call between a VOIP phone and a cell phone. I unplugged the WAN1 link and the call continued. I reconnected WAN1 and unplugged WAN2, and the call continued. This is something I did not think was possible with Meraki. We did notice brief interruptions due to lost packets. But it was much better than expected. I think the Meraki team is missing out on an opportunity here. The Velo Cloud sales reps kept telling us how VOIP calls don't drop with Velo Cloud, but they do with Meraki. I was on a call with Meraki sales reps telling them I didn't think Meraki was "real SD-WAN", and they had no response. And that's the sentiment I got from third party sales reps that sell both. They told me if I wanted true SD-WAN, to get VeloCloud. If I wanted a more reasonably-priced solution, get Meraki. One VeloCloud sales rep used the phrase "you get what you pay for". However, it is possible with Meraki to do a spoke failover without VOIP calls dropping. If they wanted to, the Meraki team could implement packet duplication and jitter protection, and compete with the VeloCloud VOIP feature set.
... View more
I've been struggling with this for the past few days. The company has VOIP telephone service. Each branch has two WAN uplinks. If an uplink goes down, then the phone calls will drop when the branch MX switches to the second uplink. If I can route the VOIP traffic to the MX at the data center, then the branch uplinks can switch without the VOIP service seeing a change in the IP address. I can also simplify IP whitelisting with 3rd party services if I can backhaul all of the traffic to those services through the hub WAN IP address. But I can't send all of the branch internet traffic to the hub because that would kill the hub's uplink bandwidth. I opened a support case and the rep responded it is not possible to set this up. I called in and the second rep told me if I can get the hub MX to advertise the route, the branch MX will send the traffic over the VPN tunnel. When I asked how to do that, he told me to put a static route into the hub MX with the IP of interest as a /32 subnet and the MX IP as the next hop. I tried that, but the traffic got into a routing loop. One would expect this to happen, but I have seen the Meraki dashboard have special support to "do what I mean" with some things like this. To resolve the routing loop, I disabled the static route. Then something amazing happened. Things started operating exactly as I wanted. The hub MX continued to advertise the route to the branch MX, and the branch MX happily sent it the traffic, but when the traffic got to the hub, the hub MX sent it out the WAN link. Perfect! So, this is great. But I am worried that this behavior would be considered a bug that will eventually be "fixed". If I were a programmer of the MX firmware, I would not advertise static routes that are disabled. The point of disabling something is to completely remove it from use without losing the information of what used be active.
... View more