After setting the option to Exclude uplink interfaces, the client list seems to be more in order. When I click on the details of the client, I get the correct device and port.
However when check the details of either the upstream or down stream switch, the bridge ports still show a ton of clients on it, but I can just ignore that port in that view. The rest of the clients are OK.
This should be good enough for my purposes since I would be looking up a client and checking where they are located.