An image processing system provides faster than real-time skin detection and localization. The system uses the highly optimized architecture of a graphics processing unit to quickly and efficiently detect and locate skin in an image. By performing skin detection and localization on the graphics processing unit, the image processing system frees the main system processor to perform other important tasks, including running general purpose applications. The speed with which the image processing system detects and localizes skin also facilitates subsequent processing steps such as face detection and motion tracking.