A time authentication system provided can detect an alteration of the case, in which either of the speakers of a conversation makes the alteration, and can verify, in the case of no alteration, that a conversation voice has occurred at a time within a range. When a speaker #1 terminal (2) and a speaker #2 terminal (3), i.e., all the speakers of the conversation request a time authentication station server (1) for the issue of a unique background sound which is difficult for those other than the time authentication station server (1) to estimate beforehand, the time authentication station server (1) issues the background sound and an electronic verification, which is electronically signed by adding a time stamp to the hash of the background sound, to the speaker #1 terminal (2) and the speaker #2 terminal (3). All the speakers of the conversation start the record of the conversation voice containing the background sound of all the speakers of the conversation, by using the speaker #1 terminal (2) and the speaker #2 terminal (3), mix and transmit their own voices while reproducing the background sound, end the record of the speech voice, and store the voice data.