A user-wearable device includes a housing and a band that straps the housing to a portion of a user's body (e.g., wrist). One or more skin contact sensors in and/or on the housing can sense biometric information of a user wearing the device. An authentication module performs or receives results of an authentication determination that compares the sensed biometric information to baseline biometric information to determine whether they match. An on-body detector uses one or more of the sensors to determine whether the device is being worn by a user. After a user is authenticated based on a match between the sensed and baseline biometric information, the authentication module continually concludes that the user is authenticated for at least a period of time, without an additional comparison between sensed and baseline biometric information, if the on-body detector detects that the user-wearable device is still being worn by the user.