A user adaptive speech recognition method and apparatus are provided. A speech recognition method includes extracting an identity vector representing an individual characteristic of a user from speech data, implementing a sub-neural network by inputting a sub-input vector including at least the identity vector to the sub-neural network, determining a scaling factor based on a result of the implementing of the sub-neural network, implementing a main neural network, configured to perform a recognition operation, by applying the determined scaling factor to the main neural network and inputting the speech data to the main neural network to which the determined scaling factor is applied, and indicating a recognition result of the implementation of the main neural network.