A machine learning device that performs reinforcement learning for a servo control device and optimizes a coefficient of a filter for attenuating a specific frequency component provided in the servo control device includes a state information acquisition unit which acquires state information that includes the result of calculation of at least one of an input/output gain of the servo control device and a phase delay of input and output, the coefficient of the filter and conditions, and an action information output unit which outputs, to the filter, action information including adjustment information of the coefficient. A reward output unit determines evaluation values under the conditions based on the result of the calculation to output, as a reward, the value of a sum of the evaluation values. A value function updating unit updates an action value function based on the value of the reward, the state information and the action information.