A computer-implemented method, computer program product and computer system (100) for determining the impact of herbicides on crop plants (11) in an agricultural field (10). The system includes an interface (110) to receive an image (20) with at least one crop plant representing a real world situation in the agricultural field (10) after herbicide application. An image pre-processing module (120) rescales the received image (20) to a rescaled image (20a) matching the size of an input layer of a first fully convolutional neural network (CNN1) referred to as the first CNN. The first CNN is trained to segment the rescaled image (20a) into crop (11) and non-crop (12, 13) portions, and provides a first segmented output (20s1) indicating the crop portions (20c) of the rescaled image with pixels belonging to representations of crop. A second fully convolutional neural network (CNN2), referred to as the second CNN, is trained to segment said crop portions into a second segmented output (20s2) with one or more sub-portions (20n, 20l) with each sub-portion including pixels associated with damaged parts of the crop plant showing a respective damage type (11-1, 11-2). A damage measurement module (130) determines a damage measure (131) for the at least one crop plant for each damage type (11-1, 11-2) based on the respective sub-portions of the second segmented output (20s2) in relation to the crop portion of the first segmented output (20s1).