A client device stores a user-specified privacy setting regulating communication of information associated with audio data captured by the client device to an external entity. When the client device captures audio data, the client device determines whether the user-specified privacy setting authorizes communication of data associated with the captured audio data to an external entity. The privacy setting may identify specific external entities to which data may be communicated, specify characteristics of captured audio data authorized to be transmitted, or generally specify whether communication of data associated with captured audio data is authorized or prevented.