I'm part of a team involved in a WAN upgrade project. We've been replacing EOL Cisco routers with Meraki MX67s. Auto-VPN is used to connect the sites to a central hub. There is an MX100 at the hub. At multiple locations, immediately after installing the MX67, Ricoh scan to email started to fail. However, at some other locations, it continued to work. Technicians have done an A:B comparison of the Ricoh settings between the sites that work and the sites that don't and haven't found any obvious differences. The Ricoh scan settings are configured to use an internal email server located at the hub site connecting via plain non-SSL SMTP over port 25. The data only has to travel over the site-to-site VPN tunnel. It never touches the internet. The Ricoh logs have generic errors that you'll find years worth of posts about in an internet search. Many of those have to do with authentication problems with Office 365 or Google G-Suite. That's not the case here. The email server is happy to accept unencrypted, unauthenticated communication. Here is an example from the Ricoh log: #[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (501) ERR:
#[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (701) ERR:
#[dcs_nas(104)]20/09/21 09:31:04 SMTPC: connection closed. (801) ERR: In the Exchange SmtpReceive log, we'll see something like the following: 2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,0,10.10.1.103:25,192.168.2.99:61118,+,,
2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,1,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.484Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,2,10.10.1.103:25,192.168.2.99:61118,>,"220 EXCHANGE1.domain.local Microsoft ESMTP MAIL Service ready at Mon, 21 Sep 2020 10:23:21 -0400",
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,3,10.10.1.103:25,192.168.2.99:61118,<,HELO RNP5838793490DB,
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,4,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.531Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,5,10.10.1.103:25,192.168.2.99:61118,>,250 EXCHANGE1.domain.local Hello [192.168.2.99],
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,6,10.10.1.103:25,192.168.2.99:61118,<,MAIL FROM:<donotreply@domain.com>,
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,7,10.10.1.103:25,192.168.2.99:61118,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,8,10.10.1.103:25,192.168.2.99:61118,*,08D85DF4C6DA20F2;2020-09-21T14:23:21.484Z;1,receiving message
2020-09-21T14:23:21.562Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,9,10.10.1.103:25,192.168.2.99:61118,>,250 2.1.0 Sender OK,
2020-09-21T14:23:21.625Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,10,10.10.1.103:25,192.168.2.99:61118,<,RCPT TO:<john.doe@domain.com>,
2020-09-21T14:23:21.625Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,11,10.10.1.103:25,192.168.2.99:61118,>,250 2.1.5 Recipient OK,
2020-09-21T14:23:21.671Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,12,10.10.1.103:25,192.168.2.99:61118,<,DATA,
2020-09-21T14:23:21.671Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,13,10.10.1.103:25,192.168.2.99:61118,>,354 Start mail input; end with <CRLF>.<CRLF>,
2020-09-21T14:23:21.718Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,14,10.10.1.103:25,192.168.2.99:61118,*,,Proxy destination(s) obtained from OnProxyInboundMessage event
2020-09-21T14:23:21.875Z,EXCHANGE1\Ricoh Test EXCHANGE1,08D85DF4C6DA20F2,15,10.10.1.103:25,192.168.2.99:61118,-,,Remote(ConnectionReset) We see that communication starts, the recipient and sender are accepted, and then the connection resets after the Ricoh device is prompted for the data. In the Exchange SmtpSend log, the following is found: 2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,223,10.10.1.103:16585,10.10.1.104:2525,>,XPROXYFROM SID=08D85DF4C6DA20F2 IP=192.168.2.99 PORT=61118 DOMAIN=RNP5838793490DB SEQNUM=1 PERMS=1073 AUTHsrc=Anonymous,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,224,10.10.1.103:16585,10.10.1.104:2525,<,250 XProxyFrom accepted,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,225,10.10.1.103:16585,10.10.1.104:2525,*,,sending message with RecordId 0 and InternetMessageId <20200921093103GH.DCSML-S000010000.5838793490DB@192.168.2.99>
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,226,10.10.1.103:16585,10.10.1.104:2525,>,MAIL FROM:<donotreply@domain.com> SIZE=0 AUTH=<>,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,227,10.10.1.103:16585,10.10.1.104:2525,>,RCPT TO:<john.doe@domain.com>,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,228,10.10.1.103:16585,10.10.1.104:2525,<,250 2.1.0 Sender OK,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,229,10.10.1.103:16585,10.10.1.104:2525,<,250 2.1.5 Recipient OK,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,230,10.10.1.103:16585,10.10.1.104:2525,>,DATA,
2020-09-21T14:23:21.718Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,231,10.10.1.103:16585,10.10.1.104:2525,<,354 Start mail input; end with <CRLF>.<CRLF>,
2020-09-21T14:23:21.875Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,232,10.10.1.103:16585,10.10.1.104:2525,*,,Proxy layer started discarding data. Acking message as failed.
2020-09-21T14:23:21.875Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20B3,233,10.10.1.103:16585,10.10.1.104:2525,-,,Local
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,122,10.10.1.103:16982,10.10.1.103:2525,*,,Proxying inbound session with session id 08D85DF4C6DA20F4
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,123,10.10.1.103:16982,10.10.1.103:2525,>,RSET,
2020-09-21T14:23:22.890Z,Inbound Proxy Internal Send Connector,08D85DF4C6DA20E9,124,10.10.1.103:16982,10.10.1.103:2525,<,250 2.0.0 Resetting, The line of interest there is "Proxy layer started discarding data. Acking message as failed." So the Exchange server is failing the message after it receives what it considers junk. Since this is all internal traffic, Meraki Advanced Malware Protection (AMP) shouldn't be involved, but could one of the Merakis be altering the traffic? I found a suggestion online to disable "SMTP inspection" if using a Cisco ASA. Obviously, nothing like that is present in the Meraki cloud controller, but is there a similar feature running on a Meraki MX? I did try opening a support case with Meraki, and the support rep did packet logging while an end user ran some scans. The technician didn't see anything of note. But the fact is that at some locations before Meraki MX67, scan to email worked. And after it didn't. Any suggestions you might have would be greatly appreciated.
... View more