Verifying Firewall Rules

[Gordon]

I opened a ticket with support and their answer does not make sense.  I have been working on tightening up my firewall rules.  In doing so I noticed I had two rules that were not showing any hits and I know there should be.  They are two rules allowing my log messages to go to my syslog server and they come different subnets.   The response I got back was that the hit counter was not reliable to use my syslogs.   The problem is the logs do not tell me which firewall rule triggered the log entry.   So now my question becomes - how do I know that the rules are working the way I think they are supposed to be.  It wouldn't be the first time I created a rule and then realized it wasn't exactly what I expected or wanted.  So if I am not getting hits on those rules how do I know it is not another rule further down in the list that is allowing the traffic.   This seems to be a big issue not being able to verify that your rules are working the way they are supposed to.

[Adam2104]

The only way to see what firewall rules are getting hit is to configure a syslog server (Network Wide ---> General) and turn on the "flows" option. After that check the syslog checkbox in the firewall rule. Sadly, there's no other way to do it. To figure out which rule is generating the syslog message you need to look at the pattern show in the syslog message itself, it should match whatever you configured on the rule.

[Gordon]

The issue is that it doesn't verify that the rule was the one that generated the log entry.  It just says "a" rule in the list generated the log entry.  So if you have a mistake in your rules you have no way of checking it to make sure it is the correct rule.  

[Adam2104]

Agreed, it doesn't tell you "rule 25 generated this log". You're going to have to determine that from the rest of the message contents. For example:

1589984219.325305969 mx67 flows src=192.168.1.82 dst=192.168.40.254 mac=AA:BB:CC:DD:EE:FF protocol=tcp sport=46928 dport=853 pattern: deny tcp && (dst port 53 || dst port 853)

Specifically, you need to look at the pattern. This will match what you've configured in your rules. The one from the above example is a Deny rule that blocks all TCP traffic to destination ports 53 and 853. Presumably you don't have several duplicate rules with the same matching criteria.

[Gordon]

I understand that.  But what if there is a bug in their firewall.  We have no way of knowing.  I already know there is one bug because the rule I have specifically for syslog does not generate any hits and there are literally thousands of packets that should be hitting that rule.  So if there is one bug there is a good chance there is another.   I also realize that with any device or software there can be bugs but without a way to verify it is not good.

[Adam2104]

Well, you could make that argument about anything it does. There's no way for you to get into the MX to see the actual ruleset that's configured. If that's a requirement for your environment you may want to consider a different product/solution that offers you that level of control / visibility.

Regarding your rules not generating hits. Are you saying in the Dashboard? If so, I've found those counters to not be real-time. The logging option on the rule is your best bet. However, I assume that is probably rate-limited to avoid causing a DoS on your own equipment by trying to log too much info too fast. Even regular big Cisco IOS/IOS-XE routers have a logging rate and queue limit.

[Nash]

---

@Adam2104 wrote:

Regarding your rules not generating hits. Are you saying in the Dashboard? If so, I've found those counters to not be real-time. The logging option on the rule is your best bet. However, I assume that is probably rate-limited to avoid causing a DoS on your own equipment by trying to log too much info too fast. Even regular big Cisco IOS/IOS-XE routers have a logging rate and queue limit.

---

Agreed that they don't seem remotely real-time. I use incrementing counters, after I'm sure I've hit save on my new rule, as a sign of life. Not any way to estimate traffic.

If you don't hit save on your new rule, well, the counters don't always show up next to the rule they're actually incrementing on... Which is fine. It's a reminder to save my rules.

[Gordon]

Yeah.  This just doesn't work for me.   I have about 60 rules on my firewalls.   So when things are not working right I need to know why.   We have a number of subnets on our network plus a number of clients that connect directly to us with a large number of subnets.  So it is essential that I can verify that my firewall rules are working as they are.  This means I need to be able to look at a rule and see what traffic it is allowing or blocking.   Seeing it on my syslog server without any way to determine which rule it came from is not acceptable to us or our clients.

To me this is a huge security issue.  I need to positively verify a rule is working as intended.

[Adam2104]

I'm sorry, but I don't understand this statement:

"Seeing it on my syslog server without any way to determine which rule it came from is not acceptable to us or our clients."

It says, in the syslog message, which pattern it came from. Sure, it doesn't say "Rule 15", but it does give you enough information to determine what rule it is from. It's not ideal, but the information is available to you.

[Gordon]

We have 8 internal subnets.  Plus we have 30 routed subnets from our clients.   These are controlled by approximately 60 firewall rules.   It is very easy to make a mistake and end up with traffic going unintended places.  Add to that our syslog server is handling over 300,000 message an hour.   It is very easy to miss something.  I have worked with a number of different firewalls and this is the first time I have had a firewall that I could not through a report, the dashboard find see what traffic was being handled by which firewall rule.

[CptnCrnch]

Yeah, Meraki MXs‘ strength is definitely not useful logging. I‘m currently setting up Splunk as log solution and building Dashboards etc. is definitely harder than with e.g. ASA or Firepower.

On the other hand: if you‘re looking for something you possibly have missed, you‘d probably better be off with some kind of network anomaly detection like Stealthwatch. I‘ve seen too many companies running state of the art firewalls, IPS, SIEM etc., but even with a few days of Stealthwatch PoC, they‘ve seen „things“ they‘d never had expected.

Especially when talking about those log volumes you‘re mentioning, it‘s completely impossible to not miss something as a human being.

[Gordon]

Well I finally found out how to verify the firewall rules.  It is possible.   The syslog entry contains a keyword called pattern.  After pattern it details the firewall rule that applied to the log entry so you can match it against the actual rule.  It would have been nicer to have something like rule ## but I can work with this.   

[phelpsa06]

This is by far my biggest drawback with Meraki, otherwise I use these where I can.   I've learned to utilize the syslog functions (we use Rapid7.)  It would do so much for Meraki to be able to show a live monitor (like Palo, Fireppower and others...)

With over 45 Meraki sites, it log fills up quick!

Not sure if it would ever be in the cards... But maybe, just maybe. lol

-my 2 cents.