An apparatus includes a filter circuit and an encoder. The filter circuit may be configured to (i) calculate a noise level of a frame of a video signal based on a subset less than all of a plurality of blocks of the frame, where each of the blocks in the subset may have an associated weighted activity to variance ratio that exceeds a threshold value and (ii) generate a filtered video signal based on the noise level. An estimated noise level of each of the blocks in the subset may be selected from one of (a) an activity of the block, (b) another activity of a reference block of another frame and (c) a sum of absolute pixel differences between the block and the reference block. The encoder may be configured to generate a bitstream by encoding the filtered video signal.