The present disclosure provides systems and methods for image-based free space detection. In one example embodiment, a computer-implemented method includes obtaining image data representing the environment proximate to the autonomous vehicle, the image data including a representation of the environment from a perspective associated with the autonomous vehicle. The method includes reprojecting the image data to generate a reprojected image. The method includes inputting the reprojected image to a machine-learned detector model. The method includes obtaining as an output of the machine-learned detector model, object data characterizing one or more objects in the environment. The method includes determining a free space in the environment based at least in part on the object data.