Does the Remote Access Key have to remain plugged in to get remote access.

Yes, if the Remote Access Key is unplugged, the remote connection will be disconnected and the user console will display a message indicating that the Remote Access Key has been disconnected.