The present disclosure provides systems and methods that measure soil roughness in a field from imagery of the field. In particular, the present subject matter is directed to systems and methods that include or otherwise leverage a machine-learned clod detection model to determine a soil roughness value for a portion of a field based at least in part on imagery of such portion of the field captured by an imaging device. For example, the imaging device can be a camera positioned in a downward-facing direction and physically coupled to a work vehicle or an implement towed by the work vehicle through the field.