A method for classifying objects in motion that includes providing, to a processor, feature data for one or more classes of objects to be classified, wherein the feature data is indexed by object class, orientation, and sensor. The method also includes providing, to the processor, one or more representative models for characterizing one or more orientation motion profiles for the one or more classes of objects in motion. The method also include acquiring, via a processor, feature data for a target object in motion from multiple sensors and/or for multiple times and trajectory of the target object in motion to classify the target object based on the feature data, the one or more orientation motion profiles and the trajectory of the target object in motion.