A system for assisting in aligning a vehicle for hitching with a trailer includes a vehicle steering system, a detection system outputting a signal including scene data of an area to a rear of the vehicle, and a controller. The controller receives the scene data and identifying a trailer and at least one ground surface within the area to the rear of the vehicle and differentiates between first and second portions of the ground surface having first and second ground surface classifications. The controller further derives a backing path to align a hitch ball mounted on the vehicle to a coupler of the trailer to maintain the vehicle within the first portion of the ground surface having the first ground surface classification and controls the vehicle steering system to maneuver the vehicle during reversing along the backing path.