A system, a computer program product, and method for controlling synthesized speech output on a voice-controlled device. One or more sensors are used to detect whether one person or more than one person is within a first settable distance from the voice-controlled device. Next a determination is made whether the audio input received is recognized as speech. In response to only one person being detected within the settable distance, begin outputting synthesized speech based on the audio input without waiting for an attention word to be recognized and otherwise wait for additional criteria before outputting synthesized speech based on the speech input. The additional criteria includes determining that more than one person is detected and recognizing that the attention word is received before outputting synthesized speech based on the audio input. Another criteria includes more than one person being detected and determining that the person is looking at the voice-controlled device before outputting synthesized speech based on the audio input.