A speech recognition system prompts a user to provide a first utterance, which is recorded. Speech recognition is performed on the first user utterance to yield a recognition result. The user is prompted to provide a second user utterance, which is recorded, processed and compared to the first utterance to detect a plurality of acoustic differences for each acoustic parameter. The acoustic model used by the speech recognition engine is modified as a function of the acoustic difference.