You appear to be running foul of the CAC time, which under certain circumstances can extend to 240 minutes.
Excerpts from my working notes -
SPECTRUM MANAGEMENT SERVICES
Because the 5 GHz spectrum used by Wi-Fi equipment is also used by others, defence, satellite communications, the maritime and aviation sectors, and weather stations when spectrum managers had anticipated they would move to other parts of the spectrum, steps had to be taken to allow this portion to be shared.
Therefore regulations were introduced that require Wi-Fi access points to
- change channels if other users (weather radar) are detected (DFS)
- reduce transmit power if other users (satellite communications) are detected (TPC)
- check that an alternative channel is not being used before using or switching to it (CAC)
- make uniform use of the available channels (DFS)
CHANNEL AVAILABILITY CHECKING (CAC)
Before a device can commence operating on a channel which requires the Channel Availability Checking function to be active, it has to listen for a set period to ensure that the channel is not already in use.
PARAMETER | VALUE |
Channel Availability Check Time | 60 seconds |
Minimum Off Channel CAC Time | 6 minutes |
Maximum Off Channel CAC Time | 4 hours |
Channel Move Time | 10 seconds |
Channel Closing Transmission Time | 1 second |
Non-Occupancy Period | 30 minutes |
NOTE 1 For channels whose nominal bandwidth falls completely or partly within the band 5 600 MHz to 5 650 MHz (Channels 120, 124 & 128) the Channel Availability Check Time shall be 10 minutes
NOTE 2 For channels whose nominal bandwidth falls completely or partly within the band 5 600 MHz to 5 650 MHz the Off Channel CAC Time shall be within the range 1 hour to 24 hours
Table 6 Channel Availability Check Parameters - ETSI EN 301 893 V1.8.0 (2015-01)
In the event that an access point is required to switch channels, all devices communicating with the access point are supposed to stop transmitting until advised by the access point to resume transmissions on the new, and available, channel. This channel will continue to be used until a radar system using the channel is detected, in which case the access point will find another available channel that is free of radar signals. The access point uses the protocols set out in IEEE802.11h to advise the connected client devices (described as slave devices by the spectrum management organisations) that transmissions should cease and that they should not transmit until advised that a clear channel has been found.
I have pages of notes relating to this. There are good reasons to reassess use of 80MHz wide channels as not all client devices can make use of the additional bandwidth.
I don't know what domain you are in, but would suggest investigating legal use of the spectrum above the DFS affected channels. Life is going to get worse, not better.