Unable to fetch configuration

[lpopejoy]

Has anyone else seen this issue?  This is the 2nd time we have seen the problem in a week. 

We factory rest an MX because of it, but even that didn't fix the problem.  Support had to "reset" something on their end to resolve.  I can tell you it's a real pain to have to factory reset a 48 port switch in production because of some phantom problem like this.  Not feeling particularly endearing.

[lpopejoy]

...and for the record, this is NOT an issue with internet connectivity. 

[NolanHerring]

I've had this happen on a couple of access points (out of like 2000 for size reference), so its not super common but I too had to have support 'reset' something on their end to resolve.

[MacuserJim]

Is it on the same device as before? I've only seen it a few times and with the number of devices I manage it's been a really small issue and a factory reset has always taken care of it for good.

[lpopejoy]

No, it was switch and an MX.  Two separate sites.

That's out of an install base of maybe 250 devices total - so still relatively infrequent.  But we haven't seen the issue in the past 3 years, and then we see it twice in a week.

[PhilipDAth]

I recall that some of the very recent firmware notes made references to resolving an issue with devices not being able to retrieve their configs.

[PhilipDAth]

Here it is, in 14.35 for the MX:

"Resolved an issue that could result in devices failing to pull configuration updates from the Meraki Dashboard in rare circumstances"

[lpopejoy]

Interesting, that would explain the MX as it was running 14.35.  It doesn't explain the MS though, but maybe we have to just accept it and move on.

[pbarbieri]

It is absurd that Cisco is not able to remove this issue ... many time i have to open a ticket to ask to fix the problem in the backend... this solution is become unreliable. I have more than 7 meraki w64 hub and spoke architecture for a mission critical application and cisco do not release a firmware to remove it. Mainly this happens as soon as power go off and meraki start again without a warm reboot and i noted that in the previous firmware release this didnt happen. 

[JohnOetinger]

we're an MSP with 140 MX64s in production. I've had the unable to fetch config issue on 5 in the last 5 weeks. You have to call support to get them to make a backend change, but that's just a temporary fix. Upgrading to 15.40 beta fixes most of the causes according to support. I am not real happy with having 2 bad options: risk customer outages or run beta, but I have chosen to upgrade all to 15.40, and so far, no problems. Supposedly this problem can occur with any 14.XX version, so I've posed the question: why is this suddenly happening now, when we've been running 14.XX on these MX64s for 2 years? no answer yet; I have a rather extensive ticket open with Meraki trying to get answers and some customer sat, which is helping, but so far has not answered all my questions.

[RupertWever]

I can confirm I’ve resolved this with a couple of client MX devices by ensuring that at least the secondary DNS is google (8.8.8.8). Sometimes the local or ISP DNS does not quickly resolve the cloud so MX cannot connect to the cloud. When using 8.8.8.8, it can find the cloud faster. You might be able to turn that back off once the connection to the cloud is reached. Don’t let downstream devices properly connecting distract you into thinking all is good - there is a difference between administration dashboard & payload routing. 

[tyoung]

The secondary DNS of 8.8.8.8 worked for me.  Thanks!  In this case it was a new firewall that was connected and disconnected, but never configured.

[pbarbieri]

The same for me unable to fetch configuration. I had this problem four time in one month. They said that is needed a new release of firmware and the beta has. Imagine i use meraki for a mission critical application

[DJDonovan · Accepted solution]

Hey everyone!

This particular error needs Support's assistance to resolve, so if you see it we definitely encourage you to reach out to Support for further assistance. Contact information for Support can be found through Help -> Get Help on the Meraki dashboard. 

-Donovan

[Felipe1]

Hi,

My problem has been solved. With a simple support contact.

Engineer response: We have applied a backend fix to mount the storage so that MX starts fetching configuration from the cloud.

thanks

[JohnOetinger]

Felipe1: Please note that that is a temporary fix. See my notes above. The only way to ensure this issue does not recur is to upgrade to 15.42.

BTW: 15.42 was moved from Beta to Stable Release about 1 week ago.

John

[Felipe1]

John

I already updated

Thanks for the suggestions

[bmehta]

Hello,

The "unable to fetch configuration" error is a known issue within Meraki. You would need to contact Cisco Meraki technical support via email or phone for their assistance in resolving the issue.

It is recommended to be on the most recent stable firmware as occurrence of this error is more on the older firmware for MXs. Also, for MXs the recent beta firmware have fixed these issue.

Thank you

[AlexLyon]

Hello, I contact support yesterday for the same problem, it's still an issue with the most recent stable firmware 🙂

[AlphaCT]

Just reporting another instance. We too are running MX64. Meraki had to fix on back end and in essence forced us to run BETA firmware as the only viable option.

Does anyone have any technical information as to root cause?

We have other sites with this appliance and would like to have any kind of intel that will allow us to consider options.

Meraki doesn't seem to be forthcoming with the details.

[JohnOetinger]

AlphaCT: replying to this and your private inquiries:

I've been told, off the record, that the cause of most of the unable to fetch config issues is caused by the file system on the MX that stores the config, which is different than the file system that houses the OS, dismounts (why has not been shared with me), and 15.4X solves most if not all of these issues. "Unable to Fetch Config', in essence means that the config is unable to be updated between the local file system and the cloud. When support "does something on the back end", they are mounting the file system, which brings the MX back online, but does not solve the problem permanently. Meraki support has been hesitant to share these details, I suppose because if it got out to competitors or the public it would be a pretty big black eye; I get that, but there's also a need to know for those of us that sell and support Meraki gear; hard to say what the right choice is; I see both sides.

We've had no more Unable to Fetch Config since I upgraded all 145 of our customer's MX64s to 15.40 about 2 months ago, and we had 5 in pretty quick succession prior to that. I asked why this suddenly became an issue with us (all of ours had been running 14.53 for a long time prior to upgrading to 15.40) and they said it had been an issue in all 14.XX revs, and is fixed in all 15.XX; I suppose it could have been a coincidence we had 5 in about 5 weeks and none prior, but coincidences are rare, so I suspect there's something they're not telling us.

Earlier this week support automatically upgraded 62 of our 110 customer Organizations from 15.40 to 15.42. Why not the other 48 is apparently just random, according to support, and presumably they will get upgraded soon too.

There is no difference between 15.40 and 15.42 relating to the unable to fetch config issue, according to support.

The only problem we have had with 15.4X is a client VPN problem. There's a backend fix that is needed to get an MX with Client VPN to pass the Diffie-Hellman group and Cipher parts of the Trustwave PCI Compliance scans to pass. That fix is not working with 15.4X, so customers are currently able to use the Client VPN but not pass the Trustwave scans, so this is only an issue for customers that have to pass PCI Compliance scans AND need/want the Client VPN AND that you move to 15.42.

We have not had any site to site VPN problems related to the unable to Fetch Config issue, 15.4X or anything else. The resiliency, stability and ease of the Meraki site to site VPNs is amazing (especially for anyone that's been exposed to how cumbersome and finicky Cisco site to site VPNs are). I saw you said that was someone else that mentioned that, but just figured I'd clarify for anyone that comes across this post.

So, while it makes me very nervous to be running 15.4X Beta on 145 customer MX64s, so far so good (other than the Client VPN issue), knock on wood.

John

[AlphaCT]

JohnOetinger, very insighfully and you adressed a series of other questions that we put forth to Meraki.

The PCI Compliance glitch is going to be an issue for some, especially where their organization's SAP and industry require it (Health and Government). But knowing this, it allows for options to be drawn up.

We appreciate your time and contribution on this issue.

[JohnOetinger]

you're welcome.

The PCI Compliance issue is being worked on by Meraki support (I have an open case and it is with Development currently) and I expect will be fixed soon (weeks probably, not months). I'll try to remember to post an update once it's been resolved.

[athan1234]

 Hi

I have the same problem .

"Configuration is out of date

This device is unable to fetch its configuration from the Meraki cloud. This is preventing your device from updating its"

If i Understand better  the problem  is for the version

Up to date

Current version: MX 15.44

[Dan-S]

this is still a common issue as of Sept 2022,  we are on the most updated firmware. we've had about 40 instances in the last 2 months where we had to call support. 

no permanent fix in sight.