An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a gesture recognition component to determine, based upon lidar sensor data, whether a pedestrian in a driving environment of the AV is performing a hailing gesture. The AV can be configured to initiate a pickup maneuver to approach the pedestrian in response to determining that the pedestrian is performing a hailing gesture.