A method of establishing a skin color model includes the following steps. A human face detecting procedure is performed on an input image, and a human face area in the input image is circled through a selecting window. A skin color model is established by using a Gaussian probability distribution function (PDF) according to color information in the selecting window. When the skin color model established in the above step is applied to skin color detection, pixels having the skin color in the input image are detected through the skin color model and a Mahalanobis distance computing procedure.