A method of performing one-time password (OTP) authentication using a color code is provided. The method includes generating, by an OTP terminal and an authentication server, an OTP using a reference time, generating, by the OTP terminal, a color code corresponding to a binary code representing the OTP, displaying, by a display device, the color code, obtaining, by a camera, the displayed color code, decoding, by the authentication server, the obtained color code into the color code and generating a password which refers to the decoded binary code, and performing, by the authentication server, authentication by comparing the generated OTP and the password.