The technology disclosed relates to a system and method for fast, accurate and parallelizable speech search, called Crystal Decoder. It is particularly useful for search applications, as opposed to dictation. It can achieve both speed and accuracy, without sacrificing one for the other. It can search different variations of records in a reference database without a significant increase in elapsed processing time. This is accomplished by first accessing the database, and for each word representation in the database, searching an utterance for likely instances of the word representation utilizing multiple parallel processors. Even the main decoding part can be parallelized as the number of words increase to maintain a fast response time.