The behaviour you're seeing certainly isn't expected if you're physically removing the WAN 1 cable.
It should failover very quickly (usually 1-2 ping drops).
I have this setup at many sites, and it works relatively flawlessly.
Given the responses above, it sounds like you've done quite a bit of testing already.
I suggest opening a support case.