Methods and apparatus for selectively performing speech processing in a hybrid speech processing system. The hybrid speech processing system includes at least one mobile electronic device and a network-connected server remotely located from the at least one mobile electronic device. The mobile electronic device is configured to use an embedded speech recognizer to process at least a portion of input audio to produce recognized text. A controller on the mobile electronic device determines whether to send information from the mobile electronic device to the server for speech processing. The determination of whether to send the information is based, at least in part, on an analysis of the input audio, the recognized text, or a semantic category associated with the recognized text.