MS250-24P Switches Going Offline

[JarettOpp]

We have over 20 MS250-24P switches that have been deployed for about 2 years now for our internal network that connect our remote offices together. Each site has a 48 volt battery plant and rectifier to supply DC power to telecom equipment. The MS switches are powered with 1 PWR-MS320-640WAC power supply connected to an inverter that's powered by the rectifier. We've had several instances over the last year where all lights on the front of the switch will power off and the switch goes offline, but the lights on the back of the power supply are lit up and looks to be operating normal. If we either unplug the power cord from the power supply and plug it back in, or reseat the power supply, the switch will come back online. We've started cases with Meraki and both times they sent us replacement switches.

My questions are if anyone else has seen or heard of this happening, and if it's recommended to have 2 power supplies powering these switches even though we're not even close to reaching the power limit on the switches?

[BrandonS]

Putting a second power supply in at least one of the switches would be a good troubleshooting step. If it failed again you should get a clue if the trouble is the power source or the power supply/switch. 

Another option might be easier to try a different power source for at least one of the switches. 

[JarettOpp]

The first case we had started with Meraki on this issue, we replaced the inverter powering the switch and inserted a second power supply and the switch still went offline. That resulted in a switch replacement back in December of last year.

The second case we had just happened a couple weeks ago where it experienced the same issues and had a replacement switch sent to us. 

Was more so wondering if a second power supply in all of the switches would help us out, or if it isn't necessary.

[BrandonS]

Based on, "..replaced the inverter powering the switch and inserted a second power supply and the switch still went offline.." it seems the trouble may be the switch itself or software and not the power supplies or power source at all.  

Have you had a failure after replacing a switch anywhere?  If they were all purchased around the same time it is feasible there is some type of hardware defect that causes them to fail like this.

This is a rough one and I wish you luck getting to the bottom of it.

[BlakeRichardson]

@JarettOpp I am guessing mains power is not an option?  How is the 48v system recharged?

What is connected to the switches and is there a pattern when they are going offline i.e. at night because a CCTV cameras IR / heater is turning on? 

[JarettOpp]

Commercial power is connected to the rectifier which then converts the AC power to 48V DC and charges the batteries. If commercial AC power is lost, there's a backup generator to keep the batteries charged and equipment from ever going down. The inverter that only powers the switch is wired to the batteries so it should never loose power. These switches are mainly used for our internal network traffic and managing other network devices that utilize out of band management in the office. These offices don't have much for PoE powered devices. The only PoE devices that are connected are for door access readers which doesn't demand much for power. The PoE consumption at these offices are less than 20 watts out of the available 370 watts with these particular power supplies. 

[JarettOpp]

The switch that we replaced this past December is actually the switch that we needed to change out again. That was after changing out the inverter and power supply again. 

We did receive all the switches at the same time when we did the initial install. 

[BrandonS]

How is the temperature control where that switch in particular is? Just a thought, because overheating can certainly expose hardware defects and cause things like lockups. And sometimes the hardware may not really be living up to it’s published temperature tolerances. 

[BlakeRichardson]

What brand and model of inverter are you using? 

To me it sounds like a power supply issue rather than hardware. 

[Ryan_Miles]

yeah sounds like a voltage dip (or spike) freaking out the power supply maybe

[JarettOpp]

I don't see spikes or dips coming from the batteries or the rectifier as these are powering other equipment in the office and we haven't seen issues with anything else. I do see the inverter possibly causing issues if it's faulty. When we measure voltage at the power cord end being plugged into the Meraki power supply, we've measured right around 120-121 volts.

[JarettOpp]

The inverters are a Mean Well TS-700-148A

[JarettOpp]

The offices are temperature controlled with an HVAC system. The temperature should range anywhere from 65 to 75 degrees Fahrenheit depending on the time of the year.