No connection for Cisco APs after VPN breakdown

[MarcP]

Hi All,

postet this problem in the Cisco Community, but maybe someone of you guys has got an idea as well. Many of you are Cisco classic Users as well.

As rebooting the MX´s solves the problem, I feel free to post it here as well. 😉

For sure, I always forgot to do a packet capture when that happend (Last time this morning) 😞

##################

Hey Community,

I´ve got a problem which is freaking me out...

We have got several sites with Meraki "Routers" (MX65) and connected to them Cisco APs (AIR-LAP1131G-E-K9 / AIR-CAP1602I-E-K9)

The MX´s have got a IPSec Tunnel to a Fortigate 1000D Firewall (providet by external).

Always, when they do maintenance at the Fortigate FW and the Cluster does not work correctly, the site loose their VPN connection. Followed by this, the AP´s are not able to get back its connection to the Flex 5000 WLC...

The only way to solve this problem is to manually reboot the AP´s, but this would mean much trouble for the stuff on site. Or to reboot all MX´s (already have a script which is doing this for me, but its still annoying).

Has anyone had this problem or knows how to avoid this?

[BrechtSchamp]

I'm surprised that the APs need a reboot. They should continue working when they lose connectivity to the controller when in Flexconnect and just keep trying to connect to the controller. Are the tunnels coming back up automatically?

"When a FlexConnect access point can reach the controller (referred to as the connected mode), the controller assists in client authentication. When a FlexConnect access point cannot access the controller, the access point enters the standalone mode and authenticates clients by itself."

"When a FlexConnect access point enters into a standalone mode, the following occurs:

• The access point checks whether it is able to reach the default gateway via ARP. If so, it will continue to try and reach the controller.

If the access point fails to establish the ARP, the following occurs:

• The access point attempts to discover for five times and if it still cannot find the controller, it tries to renew the DHCP on the ethernet interface to get a new DHCP IP.

• The access point will retry for five times, and if that fails, the access point will renew the IP address of the interface again, this will happen for three attempts.

• If the three attempts fail, the access point will fall back to the static IP and will reboot (only if the access point is configured with a static IP).

• Reboot is done to remove the possibility of any unknown error the access point configuration.

Once the access point reestablishes a connection with the controller, it disassociates all clients, applies new configuration information from the controller, and allows client connectivity again."

Source:

https://www.cisco.com/c/en/us/td/docs/wireless/controller/8-5/config-guide/b_cg85/flexconnect.html

[MarcP]

Yes, the MX establishes the VPN connection, all good on that point.

Thats the exact process the Cisco APs do, when I have them here to configure them, Or when they are set to factory defaults in a location. Thats working very well.

But it is not acting like this in the described situation.

AP is still reachable by its static IP and broadcasting the configured SSID´s (AP configured in Flexmode, btw), but Clients are not able to authenticate.

As the MX reboot also solves the problem I´m not sure if it maybe is a fault on the MX as well(?). 

Maybe it is not able to provide the information configured in DHCP

(DHCP Option - Custom - 43 - Hex - "Value of WLC) ?

[BrechtSchamp]

When you say clients are not able to authenticate you mean after the tunnels have come back right?

[MarcP]

Yes, correct, after VPN ist established again.

Any kind of WiFi Clients...

iPad, iPone or Laptop

[BrechtSchamp]

I'd use a packet capture on an AP port to see whether it succeeds in establishing its CAPWAP tunnel to the controller, and if not, why not.

[MarcP]

Yeah, thats what I need to do next time.

But always forget this, as I´m mad and the sites are retail stores and need to get back completely online in a correct ways, asap... :S

[GIdenJoe]

Can you remotely login to the AP via SSH and run a debug when this happens?
That depends if SSH is allowed on the individual AP's.

[MarcP]

Its not active at the moment, but good hint. Can activate it and also check this.

Found an article now:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCva49651/?rfs=iqvred , seems like with our old software realease im stuck :S : SW-8.0.152.0

Newwest software for the old devices.... 

One more reason to rollout MR´s 😄

[BrechtSchamp]