Methods, computing systems and computer program products implement embodiments of the present invention that include receiving, at a first time, first video and first audio signals generated in response to a user uttering a passphrase, and receiving, at a second time subsequent to the first time, second video and second audio signals generated in response the user uttering the passphrase. Upon computing an audio temporal alignment between the first and the second audio signals and computing a video temporal alignment between the first and the second video signal, the user can be authenticated by comparing the audio temporal alignment to the video temporal alignment.