A computing device senses speech or other user input. One or more physical variable pertaining to a user of the computing device are also sensed, and respective signals are analyzed or compared to the user input. The analysis determines if the user input is likely that of an authorized user, and assigns a confidence metric to that determination. The computing device may then perform actions corresponding to the speech or user input content in accordance with the determination.