A system receives sensor data from computing devices of passengers riding in an autonomous vehicle (AV). Based on the sensor data, the system can determine a position of each of the passengers within the AV. The system determines a next passenger to be picked up by the AV. Based at least in part on the position of each of the passengers within the AV, the system can (i) select a pickup location for the next passenger, and (ii) determine a route for the AV based on the pickup location such that an open seat within the AV is adjacent to the next passenger when the AV arrives at the pickup location for the next passenger. The system can transmit data corresponding to the route to enable the AV to update a current route in order to facilitate a rendezvous with the passenger at the pickup location.