In my experience bigger size only matters when you have different port speeds and are trying to shove too much traffic into a slower port.
Our HCI vendor said we needed buffers greater than 4MB per port for a 10GbE switch and most affordable ones have about 2-3MB per 8-12 ports. The buffer only smooths massive spikes as any sustained data over a port speed will be dropped anyway. We have actually found in all cases that the affordable switches cope fine, even when two of the clients are connected at 1GbE.
If you look at the maths a 10GbE port can send 1GB per second so even a dedicated port buffer of 4MB only buffers 4 micro seconds of data.
If my answer solves your problem please click Accept as Solution so others can benefit from it.