Automatic speech recognition (ASR) systems and methods. An ASR system includes a vector memory to store a plurality of feature vectors sequentially extracted from an audio data stream. A first neural network performs speech recognition processing on feature vectors stored in the vector memory to attempt to recognize a word from a predetermined vocabulary. A second neural network controls when the first neural network performs the speech recognition processing. The first neural network is held in a quiescent state except when performing the speech recognition processing under control of the second neural network.