A speech recognition system and method thereof, a vocabulary establishing method and a computer program product are provided. The speech recognition method includes: storing a speech recognition model including speech-units and basic components of acoustic models, wherein each of the speech-units includes at least one state and each state corresponds to one of the basic components of acoustic models; receiving first and second speech signals; obtaining a speech-unit sequence of a native/non-native vocabulary from a speech-analysis and unit-expansion module; recognizing the first speech signal according to the speech recognition model and the speech-unit sequence of the native/non-native vocabulary and further outputting a recognition result; and selecting an optimal component from the basic components of acoustic models according to the speech recognition model, the second speech signal, and the word corresponding to the second speech signal, and further updating the speech-units according to the best basic component of acoustic model.