why do i bother using your service??!!!!

Johm
Just browsing

why do i bother using your service??!!!!

Why in the heck am I paying for your dang service if you can't restore our configurations from a backup?? What in the heck type of company are you running? You loose our data because someone did an update and you CANT RESTORE FROM A BACKUP!!!???   WHAT THE HECK!!!!!!!!!!  

26 REPLIES 26
BHC_RESORTS
Head in the Cloud


@Johm wrote:

Why in the heck am I paying for your dang service if you can't restore our configurations from a backup?? What in the heck type of company are you running? You loose our data because someone did an update and you CANT RESTORE FROM A BACKUP!!!???   WHAT THE HECK!!!!!!!!!!  


We take screenshots of our config - we always have from the beginning just in case something like this happened.

BHC Resorts IT Department

I agree with @BHC_RESORTS, taking screenshots of the config, while an extra step for the end user, is something that is necessary in case things like this happen. This is a limitation that is up to YOU to mitigate. Should Meraki be able to fix this? Yes. Should you be prepared to fix your equipment in the event something catastrophic happens and Meraki is unable to restore your config, so you fix it by referring to your documentation that you have had from the start? Also Yes.

Found this helpful? Give me some Kudos! (click on the little up-arrow below)
ensign_fodder
Conversationalist

I am a little late to this thread, but I did not believe the data loss was configurations but only uploaded customer items.  While I had to replace floor plans, splash pages, and other "customized" items, my SSID's, PSK's, routes, etc, never had a problem.  I would love a console "level" configuration file but the use of templates makes that something of a throwback.  Anyway, it was a frustrating experience to have to redo work and I agree it should have been handled better by Meraki, but I am not sure what you are complaining about is relevant.  I am sorry if I am wrong, but again, the loss was never presented as impacting configurations.

Look,  I have a set of standards I expect a company we are doing business with to met. You are entitled to have your own set of standards. If mine are higher so be it.  I'm not going to debate the merits of paying for cloud services and what my expectations should or shouldn't be.  

Anyways, thanks for all of your suggestions.

BHC_RESORTS
Head in the Cloud


@Johm wrote:

Look,  I have a set of standards I expect a company we are doing business with to met. You are entitled to have your own set of standards. If mine are higher so be it.  I'm not going to debate the merits of paying for cloud services and what my expectations should or shouldn't be.  

Anyways, thanks for all of your suggestions.


We take multiple backups of our configurations even for non cloud based products. We don't use any Meraki switching products - yet we keep both a config backup and a text dump of the config for all our Procurves. Backups should be in your hands, not a vendors.

BHC Resorts IT Department


@BHC_RESORTS wrote:


Backups should be in your hands, not a vendors.




@BHC_RESORTS once again couldn't agree with you more. You hit the mark!

 

@Johm this is not about having higher standards than someone else. This is about understanding the limitation of the product you're using. When we were demoing Meraki, this limitation was clearly explained to us. At that point we could have not gone with Meraki or adapt to the limitation and find a work around until it was resolved. Yes it is frustrating, but to blame Meraki for not having your backup is ridiculous.

 

Many would agree with you that there does need to be some sort of way to restore the config, but until that is an option, it is up to YOU. Even if and when this becomes an option, I would tell you right now that you need to have a local copy of your configuration. What happens if that cloud backup goes missing? How are you going to recover if you are slowly relying on that cloud backup.

 

This is not me being a Meraki fan boy, this is a fellow tech talking to you. I hope you learn something from this experience and that it serves as a "take it from me experience" for other people on the forum. If you don't know what the 3-2-1 Backup Strategy is, I strongly recommend learning and implementing it. 

Found this helpful? Give me some Kudos! (click on the little up-arrow below)

Other than screen shots, I guess we don't have any other means of backing up Meraki configs as such?

Not that I know of.

 

It would be good to have some XML export / import facility, but any updates are from the recommendations of the community.

 

Thank you,
Peter James

In theory with the API you should be able to create backups of your config. No?

I’d also love to see a config export and import functionality.

Of course the API can do this, but it would be great to have this as a HTTP up- / download feature in the dashboard.

Ideally for each network and for the whole org.

Fruju
Here to help

What devices are you using? Is this even Meraki related? All Meraki configurations are stored on the Meraki cloud. It's pretty hard to lose it unless you delete all configurations from your devices.

BHC_RESORTS
Head in the Cloud


@Fruju wrote:

What devices are you using? Is this even Meraki related? All Meraki configurations are stored on the Meraki cloud. It's pretty hard to lose it unless you delete all configurations from your devices.


Except for the large service disruption recently which caused a lot of information to be lost.

BHC Resorts IT Department

Bugger. Had no idea that had occurred. Which regions got hit?

PeterJames
Head in the Cloud

I would highly recommend subscribing to the Meraki Blog.

 

This occurred 3rd August and was sent out in the blog email on 24th August. 

 

https://meraki.cisco.com/blog/2017/08/mea-culpa-and-what-happens-next/

 

To answer your question, it comes from the second paragraph...

"Earlier this month, we accidentally deleted customer-uploaded data for some of our North American users. This included floor-plan, splash page, and voice assets, but didn’t affect network operation or analytics. At Meraki we care deeply about this failure and the extra work and concern this created for our users. I’m proud of the work the team has done to accurately and quickly notify users, to recover all data possible, and to make this incident as much of a non-event for our user base as possible…  and now as the work to ensure this could never happen again begins, we have time to reflect on the incident response, methodology and philosophy."

 

Thank you,
Peter James

akan33
Building a reputation

To be honest, even if we knew about it, it is very disappointing, a cloud service benefit should includes easy backup restore. If they can't also destroy your data-config as stated in the previous post it is even more scary. It is unbelievable that you need to take manual screenshots to save your configuration in 2018.   

 

It should be as easy as 'Hey Support, I have had a massive issue, please restore my backup from day x'. 

No one here is disagreeing that we should be able to restore from the cloud. I wholeheartedly agree that this NEEDS to be something we can do. The point that I and others have tried to make is simple: you need to have a copy of your config somewhere. If the Meraki dashboard were down today (meaning you could not access it via the web), would you be able to tell me exactly how your devices are configured?

One of the most important tools in our field is our documentation. Poor documentation leads to issues further down the line. We must always be in the mindset that we could lose everything at any moment. Be a Boy Scout: Be prepared.

Look up 3-2-1 backup strategy. It means having at least 3 total copies of your data, 2 of which are local but on different mediums (read: devices), and at least 1 copy offsite.

Should I be able to download a copy of my config? Yes.
Should I be able to call support to restore my config? Yes.
Is it Meraki's responsibility for you to have a local copy of your configuration? No.
Found this helpful? Give me some Kudos! (click on the little up-arrow below)
MerakiDave
Meraki Employee
Meraki Employee

Hey Team, I saw some fresh posts on this old thread, and wanted to weigh in here, and respond mostly to the "what the heck" and "what kind of company are you running" sentiments.  First, have a fresh read of this blog post and let that start to smooth things over maybe just a little.  https://meraki.cisco.com/blog/2017/08/mea-culpa-and-what-happens-next/

 

We store customer-uploaded Objects (floor plans, custom logos on splash pages, etc) in a separate service from network configuration data due to the nature of the files themselves.  Those data stores AND their backups were accessible by a single back-end system and it was during routine engineering maintenance over a weekend that the accidental deletion happened.  This separate system along with its backups is no longer accessible from a single interface, and has been updated accordingly, such a data loss cannot happen again.  

 

While these comments will be my own and not Meraki's, I believe this was actually a failure worth celebrating.  The bad news is this caused headaches for a lot of customers, but the good news is Meraki handled this in a pretty incredible way.  If there is any silver lining, it’s that the team responded quickly, took full ownership, notified customers via website/dashboard/emails, identified which assets customers lost, and built tools to restore them in a matter of days.  A new link appeared in dashboard shortly after the announcement that allowed customers to view exactly which assets had been deleted, and we quickly built tools to make replacing these files as simple as possible.  The data cache and checksum match tools for bulk file re-upload worked brilliantly.  All was right in most customer's worlds within a couple weeks. 

 

Very few vendors could have recovered a fumble that large and complex in that time frame.  I was proud to be part of the Meraki team that pulled together in a crisis the way we did.  From the sales teams and engineers in the field (like me) taking rightfully angry calls/emails from customers, to the product teams, to the Engineering and Dashboard teams building recovery tools, it became a textbook play for turning a loss into a win. 

 

We owned the problem, fully admitted what happened and were completely transparent about it.  The crisis was not averted, it hit hard, and we showed up to fix what we broke.  We put out excellent documentation and rapid updates and tools.  I and many other engineers worked many extra hours to rebuild AP floorplan maps for some customer who did not have or could not find local backup copies of their floorplans. 

 

It was a true demonstration that Meraki cares and did everything possible to ensure customer satisfaction and retain loyalty.  Yes there were some rough days and weeks and heated discussions, but I did not have a single customer relationship weakened in the end, some stayed unchanged while many others strengthened.

 

As for network configurations, none were impacted with that data loss.  All of your network configs were safe and sound and backed up all along.  In fact, there are tertiary backups in a different data center.  So there's never just one (but as many as 3) copies of your configuration in the cloud.  Even if the whole darn cloud itself was down and Dashboard was inaccessible, you lose the ability to make active changes via Dashboard, but don't lose the ability to run your network.  Meraki equipment has a dual partition NVRAM and always has both its current and last known good firmware and config file.  So with that I could argue the config is in as many a 4 places, not 3.  The whole architecture obviates the need for on-site TFTP servers or even old school physical media backups sitting on top of the equipment.

 

Also, for anyone feeling like you have to take manual screenshots as backups: If you have a "golden config" for a network and you want to be able to restore that configuration, that's simple enough to do by cloning it to an alternate network that doesn't have any physical devices in it.  Then if you need to roll back to a specific config from a certain day/time, you can move the device(s) serial number(s) from their current network to the "backup/golden" network to revert to those older configs.

 

So to the original question, why do I bother using your service... for one thing, because of what I described above.  Last fiscal year, Meraki had 57% year-over-year growth, and still sees growth accelerating.  This just does not happen in a $1.5B business in the IT industry.  Not unless there is something very special and unique about that comany and thier products and architectures.  Sure, I'm biased, but the numbers don't lie.

 

Hope that helps, just wanted to share a little insight from my side on last year's data loss.

 

@MerakiDave, thanks for your response!

 

Now, my question is as follows: I understand that there are countless backups in Meraki's data centers, but how do you address the people that want a local copy of their configuration? People take screenshots of their configs for a reason, the number one being documentation purposes. XML files have already been mentioned in this thread. I have text files of all the devices in my network. I'm sure others have other things they do.

 

Templates are great, but even templates have limitations. If there is any type of summarized route on your network and the network you are trying to create falls under one of the auto-generated networks, you can't use the template.

 

We need something that we can download to say "Yeah, that's my config"

Found this helpful? Give me some Kudos! (click on the little up-arrow below)

@Mr_IT_Guy

 

And to follow on from your point, it would be so re-assuring to be able to snapshot the configuration before making experimental changes, so that one could rollback in the event that all the changes had to be undone.

 

Robin St.Clair | Principal, Caithness Analytics | @uberseehandel

Some great ideas being thrown around here. This is ultimately coming down to security vs usability. Routers/Switches had this issue for years, until they eventually caved and allowed you to download/upload settings as required.

 

I manage a number of networks and admittedly keep the settings offline as I cant possibly remember each one, but I have kept each client estate the same with templates. Do I keep an offline copy of the settings? Yes.

 

Call me a little security gnome, but I would rather nobody see the password I have set in the interface unless I explicitly give them permission. This is something the interface can not do. But back to point....would I like someone to have the ability to download my settings in one central location? No!

 

I would rather download an encrypted settings file that I can later re-load in to the interface should Winter appear.

 

This is where I think this particular thread is splitting:
 - One group want, fully readable / downloadable settings file.

 - Another group wants an encrypted backup copy than can be reload in to the interface.

 

----

 

@Uberseehandel One thing I do is receive alerts on any network changes, so I know if someone got a bit trigger happy. It shows the old and new setting in the email. This might help you achieve this in the interim. Obviously it will also include any settings you change too.

 

Thank you,
Peter James


@PeterJames wrote:

 

@Uberseehandel One thing I do is receive alerts on any network changes, so I know if someone got a bit trigger happy. It shows the old and new setting in the email. This might help you achieve this in the interim. Obviously it will also include any settings you change too.

@PeterJames

 

That sounds like a good idea, thanks.

 

 

 

 

 

Robin St.Clair | Principal, Caithness Analytics | @uberseehandel

@Uberseehandel, keep in mind, there are some things that do not report. For instance: When you change the ports on the security appliances, you do not get the before and after.

Found this helpful? Give me some Kudos! (click on the little up-arrow below)


@Mr_IT_Guy wrote:

@Uberseehandel, keep in mind, there are some things that do not report. For instance: When you change the ports on the security appliances, you do not get the before and after.


@Mr_IT_Guy

Thanks for the heads up. At the moment, and until Meraki sorts out a functioning IGMP-proxy on the MX, I'm only using a single trunk uplink to the MX.

 

In the mean time, getting ChromeCast to work across two different VLANs, despite getting all the ports and protocols from Google, and explicitly allowing them, I feel a bit like a yo-yo. As long as the changes on the switch ports are recorded and the allowed VLANs up to the SSIDs, I'll cope.

 

Thanks again

 

 

Robin St.Clair | Principal, Caithness Analytics | @uberseehandel

@Uberseehandel regarding the "snapshot" config, that's what I meant by "golden config"... If you have a "golden config" for a network and you want to be able to archive that network configuration prior to a change, you can clone it to an alternate network that doesn't have any physical devices in it.  Then if you need to restore that specific config from a certain day/time, you can move the device(s) serial number(s) from their current network to the "backup/golden" network to revert to those older configs.

 

@Mr_IT_Guy I get your point and don't argue that value and maybe peace of mind with having a safe, local copy of config data, and it's just another part of having complete documentation.  And I don't really have a good answer other that "that's the model/architecture" at least for now, and even though I can't rattle off the exact details here, there are engineering, architecture and even regulatory & compliance reasons.  One idea for straight-up documentation, a simple and low-tech thing I did for a customer's documentation purposes, was a Dashboard walk-through of all menus and settings, on a recorded Webex session.  This also allowed me to speak and record my comments in real time while looking at each screen, and why certain settings were selected.  That was just what they needed initially, afterwards we repeated the process more formally, but also included monitoring and troubleshooting tasks, and turned it into an overview training video for new hires.  Just a thought, maybe one option versus dozens of screenshots that may not have context or correlation between them.

@MerakiDave "One idea for straight-up documentation, a simple and low-tech thing I did for a customer's documentation purposes, was a Dashboard walk-through of all menus and settings, on a recorded Webex session.  This also allowed me to speak and record my comments in real time while looking at each screen, and why certain settings were selected.  That was just what they needed initially, afterwards we repeated the process more formally, but also included monitoring and troubleshooting tasks, and turned it into an overview training video for new hires."

 

I am glad I am not the only one doing that 🙂 Just to add, sometimes just walking someone through a solution setup brings to life a problem. And if you cant justify a setup choice, normally its not required!

 


@MerakiDave wrote:

@Uberseehandel regarding the "snapshot" config, that's what I meant by "golden config"... If you have a "golden config" for a network and you want to be able to archive that network configuration prior to a change, you can clone it to an alternate network that doesn't have any physical devices in it.  Then if you need to restore that specific config from a certain day/time, you can move the device(s) serial number(s) from their current network to the "backup/golden" network to revert to those older configs.

 


Thanks for the suggestion, I'll experiment with that.

 

 

 

Robin St.Clair | Principal, Caithness Analytics | @uberseehandel
Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels