Systems and methods provide for indexing audio content by fusing the indexes derived from a keyword stream and a large vocabulary stream search. For example, systems and methods provide for two stream searching of Spoken Web VoiceSites, wherein metadata is extracted from the VoiceSite and is used to determine a set of keywords for high precision search while a traditional standard vocabulary set is used to perform a high results, low precision search. The results of the keyword search and the standard vocabulary search are fused together to form a comprehensive, ranked list of results.