This document describes techniques and systems for fuzzy labeling of low-level electromagnetic sensor data. Sensor data in the form of an energy spectrum is obtained and the points within an estimated geographic boundary of a scatterer represented by the smear is labeled with a value of one. The remaining points of the energy spectrum are labeled with values between zero and one with the values decreasing the further away each respective remaining point is from the geographic boundary. The fuzzy labeling process may harness more in-depth information available from the distribution of the energy in the energy spectrum. A model can be trained to efficiently label an energy spectrum map in this manner. This may result in lower computational costs than other labeling methods. Additionally, false detections by the sensor may be reduced resulting in more accurate detection and tracking of objects.