A speaker recognition device includes a memory, and a processor. The memory stores enrolled key phrase data corresponding to utterances of a key phrase by enrolled users,and text-dependent and text-independent acoustic speaker models of the enrolled users. The processor is operatively connected to the memory, and executes instructions to authenticate a speaker as an enrolled user, which includes detecting input key phrase data corresponding to a key phrase uttered by the speaker, computing text-dependent and text-independent scores for the speaker using speech models of the enrolled user, computing a confidence score, and authenticating or rejecting the speaker as the enrolled user based on whether the confidence score indicates that the input key phrase data corresponds to the speech from the enrolled user.