Systems, methods, devices, and techniques for generating object-heading estimations. In one example, methods include actions of receiving sensor data representing measurements of an object that was detected within a proximity of a vehicle; processing the sensor data with one or more preliminary heading estimation subsystems to respectively generate one or more preliminary heading estimations for the object; processing two or more inputs with a second heading estimation subsystem to generate a refined heading estimation for the object, the two or more inputs including the one or more preliminary heading estimations for the object; and providing the refined heading estimation for the object to an external processing system.