Technologies are described for the automated determination of materials. For example, material can be automatically identified (e.g., unique material numbers can be determined) based on sensor data and using machine learning models. In some implementations, as part of a first phase, a first set of sensor information describing the material is obtained. Using the first set of sensor information, a material class of the material is determined. As part of a second phase, a second set of sensor information describing the material is obtained. Using the second set of sensor information, the specific material is identified (e.g., a unique material identifier is determined for the material).