110 units of MX84/MX100 routers, randomly start not responding to DHCP requests

[ronnieshih75]

We have over 110 mixture of MX84 and MX100 routers, all running on v15.42.1 firmware for nearly 3 months now.  A few days ago, like a time bomb, a lot of them stopped responding to DHCP requests, this caused pc's to fall off the network with 169.254.x.x self-assigned IP addresses.  The only fix was to reboot the router, it would be fine for hours then it would start doing the same thing again, not handing out DHCP IP addresses.  Called Meraki support and was told to upgrade to v15.43.1 release candidate, ok, did that and rebooted routers, now it's doing the same thing AGAIN.  Has anyone experienced this issue?  We run clinics and literally rebooting a bunch of routers DURING THE DAY DURING BUSINESS HOURS!

And for Cisco Meraki -> stop throwing the firmware card!!!!

[DarrenOC]

You’re not alone with this one:

https://community.meraki.com/t5/Switching/MS250-not-responding-to-DHCP-requests/m-p/120568#M8804

ive also heard from peers that they’re suffering the same issue and in some cases have resorted to deploying Windows dhcp services

[ronnieshih75]

So what's the solution here?!  Everything was fine up until 2 days ago, with no change, no firmware upgrade, MX84 and MX100 routers just stop responding to DHCP requests?  If there is an infinite DHCP lease option, I would have done it already, but there is not!!  I've been rebooting routers throughout the entire day and it's just purely ridiculous.  No, I do not have VLAN tagging issue.  We run the same setup across all clinics and it's the same setup I've been doing since I joined this company.

[cmr]

No that is necessarily of help to you, but we have 9x MXs serving DHCP and so far 🤞 haven't seen this problem... 

All of these MXs are running 16.11 or 17.2 and are a mix of Z3, MX64, MX65, MX67 and MX84.  We configure either a /24 (Z3 and MX65s) or a /16 subnet and have a 4-24 hour lease time.

[ronnieshih75]

We were told to upgrade the ones with this DHCP issue to v15.43.1, made no difference whatsoever.  Just over 12 hours later, pc's started dropping off the network with 169.254.x.x IP addresses again.  So what now?  Throw another firmware card?  Bring laptop to park picnic and reboot all routers while having a sandwich?  I had no issue whatsoever until 2 days ago, then it started out of the blue with ZERO change.  Maybe I should throw my old Cisco 2600 router back in or start building Window server on pc's to serve DHCP.

[cmr]

@ronnieshih75 you have several options and I was just letting you know that there might be a different firmware that you could try as we don't experience the issue.  I have no idea if it will help your particular situation, but unless you want to keep rebooting, or add a separate DHCP server to cover each site, I'm not sure what to suggest.

Remember, we're mainly customers and integrators here, unless you see the green M on the account which indicates an employee.

[JimmyM]

Hi Ronnie

I had exactly the same issue with my MX84. After a few months of pain, i just migrated my DHCP Back to my Windows Server.

I tried to contact Meraki Support About that but never had the patience to do the entire process.

When it happen it's just to urgent to wait...

[ronnieshih75]

I will be building a windows 2016 server running DHCP and relay all DHCP requests back to headend data center instead.  But don't tell me that that even DHCP relay is busted... I hope that's not the case.  

What bothers me here is the fact that everything was fine since I joined this company more than a year ago then this problem just started a few days ago while running on the same firmware for the last 2 months and 3 weeks.