To enhance convenience in an authentication system using a plurality of types of authentication, a terminal device transmits authentication data including a face image and a voice of a user, the position of the terminal device, to the server device. The server device uses the received authentication data to perform individual authentications including face authentication, voiceprint authentication, position adequacy verification. The server device applies adds up weighted scores of the individual authentications to calculate a comprehensive score. When the comprehensive score exceeds a first threshold, a high security operation is permitted. When the comprehensive score is not higher than the first threshold and exceeds a second threshold, a low security operation is permitted. When performing additional authentication using additional authentication data received from the terminal device, the comprehensive score further includes the additional authentication score, and the high security operation is permitted when the comprehensive score exceeds the first threshold.