An adaptive voice authentication system is provided. The adaptive voice authentication system includes an adaptive module configured to compare a feature quality index of the plurality of authentication features and the plurality of enrollment features and dynamically replace and store one or more enrollment features with one or more authentication features to form a plurality of updated enrollment features. The adaptive module is configured to generate an updated enrollment voice print model from the plurality of the updated enrollment features. The adaptive module is further configured to compare the updated enrollment voice print model with the previously stored enrollment voice print model and dynamically update the previously stored enrollment voice print model with the updated enrollment voice print model based on a model quality index.