Implementations disclose mutual noise estimation for videos. A method includes determining an optimal frame noise variance for intensity values of each frame of frames of a video, the optimal frame noise variance based on a determined relationship between spatial variance and temporal variance of the intensity values of homogeneous blocks in the frame, identifying an optimal video noise variance for the video based on optimal frame noise variances of the frames of the video, selecting, for each frame of the video, one or more of the blocks having a spatial variance that is less than the optimal video noise variance, the one or more frames selected as the homogeneous blocks, and utilizing the selected homogeneous blocks to estimate a noise signal of the video.