Methods and systems for face detection and tracking using an image-based capture device are disclosed herein. The method includes generating a depth image of a scene, generating a mask image from the depth image, and detecting a position of a face of a user in the scene using the mask image. The method also includes determining an intensity of the face using a first color channel of the mask image and adjusting a gain level of a first color channel of the image-based capture device directed at the scene to achieve a target intensity of the face.