An authentication management system for managing use of a processing apparatus includes an authentication management apparatus including a position storing unit to store position information of the processing apparatus; a receiving unit to receive user information that identifies a portable terminal to be used for authenticating when a user requests the processing apparatus via a requester to perform a target process; a determination unit to determine whether distance between the portable terminal and the processing apparatus satisfies a proximity determination condition between the portable terminal and the processing apparatus, the distance being obtained based on position information of the portable terminal received from the portable terminal, and the position information of the processing apparatus; and an authentication unit to conduct authentication processing based on user information received from the requester, and the user information received from the portable terminal when the distance satisfies the proximity determination condition.