The present invention relates to a method for calculating optimal control force of an active mass damper and controlling the active mass damper, which is used for vibration control of a structure, the method comprising: (a) sensing frequency and an acceleration response generated in a structure in real time by a sensor, wherein the frequency and the acceleration response are structural responses; (b) calculating velocity and displacement of the active mass damper by integrating the acceleration response by an integral calculation module; (c) calculating the optimal control force using a weighting function and a zero point calibration signal according to the calculated velocity, the calculated displacement, and the calculated displacement, by a control force calculation module; and (d) controlling driving of an actuator according to the calculated optimal control force. Accordingly, the method/system for calculating optimal control force of an active mass damper and controlling the active mass damper according to the present invention can control a flow phenomenon of a moving mass body by a weighting function and a zero point calibration signal, enable stable control at a zero point, and remove signals other than those in a control mode through a band pass filter, thereby accurately generating a control signal and achieving performance which can satisfy a control target for each mode.