In a vehicle teleoperation session, a speed limit is determined for which the vehicle can be safely teleoperated. A safety system senses data relating to the vehicle environment and generates a depth map from which obstacles in the vicinity of the vehicle can be identified. Based on the detected obstacles and a motion state of the vehicle, a speed limit is determined at which the teleoperator is predicted to be able to utilize an emergency braking command to avoid a collision. The speed limit may be automatically applied to the vehicle or may be provided to the teleoperator to enable the teleoperator to adjust the vehicle speed.