Techniques are provided for defending against an ultrasonic attack on a speech enabled device. A methodology implementing the techniques according to an embodiment includes detecting voice activity in an audio signal received by the device and generating an ultrasonic jamming signal in response to the detection. The jamming signal is broadcast over a loudspeaker for up to the duration of the detected voice activity to defend against the ultrasonic attack. According to another embodiment, the ultrasonic jamming signal is generated in response to detection of a wake-on-voice key phrase in the received audio signal, and the jamming signal is broadcast over the loudspeaker for a time duration selected to be less than or equal to a time window during which spoken commands are accepted by the device following the wake-on-voice key phrase detection. The jamming signal may include white or colored noise, combinations of tones, and/or a periodic sweep frequency.