An embodiment of the present invention relates to a speech recognition system and method using incremental device-based acoustic model adaptation. The speech recognition system comprises a model selection module selecting an acoustic model of multi-model tree by verifying and categorizing a device key transmitted from a user device; a model management module generating and incrementally adapting multi-model tree by categorizing voice data based on a user device; and a speech recognition module performing speech recognition by receiving the acoustic model selected from the model selection module and transmitting data of which reliability exceeds a predetermined threshold value to the model management module.