cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

[WINNERS ANNOUNCED] Community Challenge: The Impossible Fix

Community Manager

Screen Shot 2019-06-07 at 12.18.56 PM.png

 

UPDATE: This contest has ended. Huge thanks to everyone for sharing your stories, we loved reading all of them! Congratulations to our three randomly selected winners: @jfry2k@MakkyCro, and @LuisCruz.   

 

For many of us, it’s been a few months since our sudden transition to the world of remote work.  We’ve had to navigate more than a few snags in our setups, such as unexplained internet outages, network security breaches, and even hardware malfunctions (maybe your keyboard and the contents of your Diet Coke became close friends). All requiring contact-free intervention.

 

For this month’s challenge, we want to know about the craziest / most  painful / most impossible fix you’ve had to resolve remotely using Meraki. (This could be a fix made during or prior to COVID)

 

Here is an array of colorful examples provided by my colleague @Phi-L, Technical Trainer to the Stars:

 

  1. Maybe an intern decided to plug in all the loose cabling they could find in a room back into a switch, introducing a loop. You used the dashboard to troubleshoot RSTP issues.
  2. You’re the dad of a wallowing son, who now is stuck back at home due to COVID closing his college. In his despair, he is sucking up all the bandwidth by streaming Twitch and playing PS4 all day. You used traffic shaping rules to deprioritize that traffic and handed him a copy of One Hundred Years of Solitude to fill his time.
  3. Perhaps someone was INSISTENT that a device on a wall was an AP, but you used the blink LEDs live tool to track down the AP he was actually looking for and prove the mystery device to be a smoke detector.

How to enter

Post a comment on this blog post containing your story about an impossible fix that you’ve pulled off remotely. Submit your comment before June 23rd at 11am PT and be entered to win a fantastic specimen of Meraki swag (a Meraki mini block set!):

 

mrkimfka0112699_6.jpg

 

 

We’ll then randomly select three winners from the list of entrants. Good luck!!

 

Fine Print:

37 Comments
Here to help

Meraki dashboard did not allow me to change STP root;  Meraki support team was professional, yet unable to perform the task either.  I happen to be working on python scripts to automate other tasks so I built a quick postman call to make the change: success!  I believe case is still open for Meraki development to investigate.

Conversationalist

Two words... 

 

Firmware Bugs.

Here to help

Meraki dashboard only allows download of network event-logs for one page at a time, and we needed longer history list of events.  So I built a python script that downloads 500 pages worth of event logs history.  Now I'm even adding additional lookups to include client vendor so we can look for patterns.

Conversationalist

Mission 1 question 3 is broken.

 

How many daily API requests are made through the Meraki dashboard?

 

I did put 45million and it marked it as wrong.2020-06-17_12-09-13.png

Conversationalist

My first installation we dont now how to begin the installation and only with the markai pageI did it

Conversationalist

I had a customer that had a MX fail and would not respond.  Luckly the customer had and old MX on site and was able to swap the units and I was able to temporarily get them up and running.  Then I did an rma of the failed unit.  When the customer received the replacement, I was able to put it back into production, with out ever going on site.

Getting noticed

We had a customer with a problem on a VPN between the US and a Canada data center. The VPN was up but traffic was dead slow through it. I used MTR to discover the issue with between two ISP's at the border, the handoff was introducing huge latency between the two. I was able to re-route traffic to another Canadian site that had a VPN back to the main Canadian hub using the Exit Hub configuration. This worked until the ISP issues were resolved a week later.

This branch would have been essentially down without this option.

 

MTR was valuable in showing both ISP's (you know the finger pointing game) where the issue was.

 

Thanks 

Here to help

Migrating the corporate wide wireless authentication and authorization infrastructure to a new ISE implementation is something that causes MANY eyes to be watching and looking.


Meraki's built in tools and packet capture capabilities allow the testing, and validation and POC's to be all done REMOTELY.

We were up against a hard deadline.

Then the current situation hit.

 

The tools in Meraki's platform allow the implementation, POST validation and POST monitoring and validation to occur without a hitch.

A few incidents tickets came in but the tools in the platform allow us to see that the issues were because the users were using improper wifi setup on the devices or bad credentials.

 

Work smarter not harder.

Thanks Meraki for letting this happen!

 

 

A model citizen

We were having issues passing PCI scans due to Meraki Client VPN.. After several call Meraki Support change the Client VPN Encryption to more stringent requirements (AES128 encryption with DH group 14 - Required by PCI-DSS 3.2). Now had to update all my end points to use this new encryption standard and configure the Meraki Client VPN... I was able to cerate a small PowerShell Script to automate this process for me and set all the settings needed.

 

$ServerAddress = "xxx.xxx.xxx.xxx"
$ConnectionName = "Meraki Client VPN"
$PresharedKey = "A Password"
Add-VpnConnection -Name "$ConnectionName" -ServerAddress "$ServerAddress" -TunnelType L2tp -L2tpPsk "$PresharedKey" -AuthenticationMethod PAP -Force
Start-Sleep -m 100
New-NetIPsecMainModeCryptoProposal -Encryption AESGCM128 -Hash SHA1 -KeyExchange DH14
New here

Got a call during holiday that the WiFi was down.
With the help of the dashboard, and the Meraki support team, we identified a firmware bug.

The happy ending of the story, even being remote, I was able to restore the WiFi only using the dashboard 🙂
Bad side of the story, ended up using a beta firmware on a production network.

Here to help

At some point the number of organizations in my dashboard, including test orgs, was over 20 so I decided to do some clean up, so I started deleting test organizations and accidentally deleted a client semi-production/testing network. Wooow! I almost die, but was able to recover 95% of the config just by following step by step the Dashboard Change-log. The client was cool about it, he just asked me "are we back up?", I said "Of course we are! this is Meraki 🙂

Getting noticed

My parents home runs Meraki. One day my mother called me because their internet had gone down. After a bit of ranting and raving about how it doesn't make sense for me to run their house on enterprise equipment when I cant be home to manage it, my mother said she would ask my older brother who lives close to them to check it out later in the day. Given that his expertise is IT management, and a lack of a login for their network, that was going to be tough. 

 

I checked their network, and saw the issue was just a cable that had been unplugged, which was strange as nobody goes near the networking stuff besides me. Texted my father and brother who would be at the house later in the day to let them know which cable it was so one of them could fix it.

 

Turns out my brother had been walking around the basement earlier that day and stepped on the cabling. 🤦‍

Here to help

be me

systems engineer at a small computer consulting company

client calls complaining they cant use vpn

try to access vpn

doesn't work because i run linux

client isn't far away

get in 2015 mustang ecoboost to speed on over

client uses shared office space

go into lobby

badge doesnt work

go to security desk

"uuuuh my badge doesnt work"

cant go in without an appointment because covid

explain that the vpn is down

still cant go in

go back to the 'stang

get an epic idea

drive around the building searching for signal

see my clients ssid thanks to the wall penetrating power of the Meraki MR33 2.4Ghz Channel

connect to network

ssh to clients asa

fix vpn

Spoiler
save the day
Comes here often

Migrating a client to a new ISP. 

 

I was able to help a client move to a new ISP fully remotely. They received all the new IP information in advance, and I was able to via VPN switch port 2 on an MX to a wan port and configure it with the new IP information. Further more, I configured new 1:1 NAT rules in advance in the dashboard and advised the client to prepare their DNS with lower TTL values to speed up the change over to new public IP addresses at the time of the cutover. The client was then able to plug in the new ISPs connection in port to, and we were able to set WAN 2 as primary uplink to migrate the client over to the new ISP.

 

The client is based in Sweden and I did this whilst locked down in my flat in the UK. 

 

 

Getting noticed

Trying to make Meraki client VPN pass PCI scans.  5 calls with Meraki support and 4 calls with AT&T support.

 

Oh wait, problem still not resolved......

 

 

TO BE CONTINUED!

A model citizen

@Jwiley78  our Meraki Client VPN passes PCI scans.  You have to have support up your encryption to AESGCM128 and KeyExchange to DH14 then change the encryption level on your end points... I did this via PowerShell 

New-NetIPsecMainModeCryptoProposal -Encryption AESGCM128 -Hash SHA1 -KeyExchange DH14

 

Getting noticed

That has been my struggle.  Changing the end points.  Thanks for the script.  I'll try that.

 

Any luck with Macs?

A model citizen

we are a pure Windows environment so I've only tried it on windows.. here is my full script I use for deployment.. I'm working on a full write up on this as we speak.

$ServerAddress = "xxx.xxx.xxx.xxx"
$ConnectionName = "Meraki Client VPN"
$PresharedKey = "A Password"
Add-VpnConnection -Name "$ConnectionName" -ServerAddress "$ServerAddress" -TunnelType L2tp -L2tpPsk "$PresharedKey" -AuthenticationMethod PAP -Force
Start-Sleep -m 100
New-NetIPsecMainModeCryptoProposal -Encryption AESGCM128 -Hash SHA1 -KeyExchange DH14

 

Getting noticed

Nice, bookmarking this for later.  I've now got a project for tomorrow.  Hopefully it can be the impossible fix that is solved by the community.  🙂

 

 

Here to help

300 WAPs in 600 seconds (60 would have been better but this will do).

 

Due to the complexity of a building site and delays by subcontractors, I decided to licence and configure the entire wireless on my mobile while watching TV in under 10 minutes.