The present invention provide a full-servo label inspection machine and a control method thereof; the machine includes a frame, the frame is provided with an unrolling mechanism, a rolling mechanism, a visual inspection mechanism, a label removal mechanism, and a label replacement mechanism. In the label removal mechanism, when the paper pressing roller is at the label removal position, it presses the material to be inspected downwards to form an angle of less than 180° with the rear end of a label removal plate. After a defective label on the material to be inspected passes the label removal plate, a front end of the defective label curls up and eventually sticks onto the label removal paper. The present invention allows fully automation of the removal and replacement of labels, achieving high inspection efficiency and accuracy.