An elevator control system (18) provides elevator dispatch and door control based on passenger data received from a video processing device (12). The video processing device (12) includes a video processor (14) connected to receive video input from at least one video camera (10) and memory (16). The video processor (14) anonymously monitors passengers using color index analysis of each passenger. Based on the monitored position of each passenger, video process (14) calculates passenger data parameters such as location, direction, speed and estimated time of arrival at an elevator door, and provides one or more of the passenger data parameters to elevator control system (18). Based in part on the passenger data received, elevator controller (18) provides elevator dispatch, elevator door control, and security functions.