A system for tracking objects using an Intelligent Video processing system in the context of airport surface monitoring. The system addresses airport surface monitoring operational issues such as all weather conditions, high robustness, and low false report rate. The output can be used to complement existing airport surface monitoring systems. By combining the use of multi-sensors and an adverse weather optimized system, the system is capable of producing an improved stream of information for the target object over traditional computer vision based airport surface monitoring systems.