A machine learning device performs reinforcement learning on a controller that performs multiple processes for controlling a machine tool in parallel at multiple operation units. The machine learning device comprises: behavior information output means that outputs behavior information containing allocation of arithmetic units that perform the multiple processes to the controller; state information acquisition means that acquires state information containing a machining condition as a condition for machining set at the machine tool, and determination information generated by monitoring the implementation of the multiple processes by the multiple operation units based on the allocation in the behavior information; reward calculation means that calculates the value of a reward to be given by the reinforcement learning based on the determination information in the state information; and value function update means that updates a behavior value function based on the reward value, the state information, and the behavior information.