A method, system and computer program product for providing video privacy is provided herein. First video data captured by a video camera is received. A context for the first video data is determined. It is determined that the context matches a privacy context from a set of privacy contexts identified using machine learning. In response to the context matching the privacy context, at least a portion of second video data is blocked that is captured by the video camera subsequent to the first video data.