Solarwinds NPM snmp polling works somewhat

[RANT]

I have Solarwinds NPM monitoring a pair of MX105 firewalls (used as inline vpn concentrators). The status of the firewalls is down, but it's able to poll the interface traffic usage. Other remote end VPN MX firewalls don't have this issue, their status is UP.

[Inderdeep]

What exactly is the issue 

https://documentation.meraki.com/General_Administration/Monitoring_and_Reporting/SNMP_Overview_and_C...

[michalc]

Hi There,

The MX105 shouldn’t just flip to "Down" out of the blue. Have it been working before? Have there been any recent changes—like a firmware update on the MX105 pair in the last week or so? Maybe something sneaky like a network tweak or a config change that could’ve thrown things off? Also, how’s the reachability looking? Can the polling engine ping the MX105, or is something like a firewall rule or routing hiccup getting in the way?

Here’s where I’d start with:

• Check the polling method: Head over to the node details in NPM, click "Edit Node," and see if it’s set to "Most Devices: SNMP and ICMP." If ICMP’s blocked or timing out, that could explain the "Down" status, even if SNMP is happily pulling traffic data.
• Run a quick test: Try a manual ping from the NPM server to the MX105 IPs. If that fails, maybe the inline setup is causing ICMP to drop somewhere along the path.
• Grab a PCAP: If you’ve got access, run a packet capture from the NPM server toward the MX105. Are we seeing SNMP responses coming back? Any signs of ICMP replies (or lack thereof)?

[RANT]

PCAP on the MX105 shows an echo request and reply. On the solarwinds server though, pings do not show a response.

[michalc]

Sounds like something else upstream from the MX105 is blocking the traffic. Do you have any other firewall upstream from MX105? Perhaps an ISP modem/router with Security features enabled?

[RANT]

The way this inline vpn concentrator is setup is weird. the default route is set not to the core switch but to the internet firewall. SO i think there is some assymetric routing happening here: ping goes from solarwinds to the MX, but the reply is sent up to the internet firewall and then back down to solarwinds.

[tnco · Accepted solution]

For an one arm concentrator, I think it is correct that the default gateway points to the FW.

It also needs to be separate from the client subnet as described in the following document.

https://documentation.meraki.com/MX/Site-to-site_VPN/One-Armed_VPN_Concentrator_Deployment_Guide

https://documentation.meraki.com/MX/Networks_and_Routing/Passthrough_Mode_on_the_MX_Security_Applian...

[RANT]

yeah I see how this works, but this creates a problem: the internet firewall is now processing the decrypted internal traffic as well, because the MX decrypts the incoming traffic, and then routes it back to the internet firewall, which now routes the traffic back (again) to the internal network. This places an unnecessary load on the internet firewall to process additional traffic.

[PhilipDAth]

How does it determine the status?  Via ping?  Can the machine ping the address you are monitoring?

[IvanJukic]

Hi @RANT ,

As with any monitoring platform out there. We need to be sure we are monitoring the appropriate Interface, Module, Socket etc. Then ensure we have the right reporting mechaismim in place. Sound like icmp ping timeouts, perhaps?

1.) Is Solarwinds telemetry polling within the VPN Domain or "Outside"?
2.) What protocol is used to poll this MX info?