Techniques for unobtrusively verifying the identity of a user of a computing device are provided. In one embodiment, the computing device can establish one or more verification models for verifying the user's identity, where at least a subset of the one or more verification models is based on enrollment data that is collected in an unobtrusive manner from the user. The computing device can then verify the user's identity using the one or more verification models.