It's simply no other way of knowing this.
You have to attempt to connect to the device to know if it's not there and it's setting up that connection that is slow.
The same applies when you have a connection. The underlying stack probably tries to avoid dropping the connection since it isn't necessarily a power down but could just as well be the remote device being in "radio shadow".
I could point out though that running the application outside of a debugger is usually a lot faster. (if you are running it from within Visual Studio)