A vision system for a vehicle includes a camera and an electronic control unit (ECU) with an image processor. The ECU generates a reduced resolution frame of captured image data and the ECU determines a reduced resolution detection result based on pedestrian detection using the reduced resolution frame of captured image data. The ECU, responsive to processing by the image processor of image data, generates a cropped frame of captured image data and the ECU determines a cropped detection result based on pedestrian detection using the cropped frame of captured image data. Responsive to determining the reduced resolution detection result and determining the cropped detection result, the ECU merges the reduced resolution detection result and the cropped detection result into a final pedestrian detection result. The final pedestrian detection result is indicative of presence of a pedestrian within the field of view of the camera.