Systems for caller identification and authentication may include an authentication server. The authentication server may be configured to receive audio data including speech of a plurality of telephone calls, use audio data for at least a subset of the plurality of telephone calls to store a plurality of known characteristics each associated with a specific demographic, and/or use audio data for at least one of the plurality of telephone calls to identify the telephone caller making the telephone call based on determining a most similar known characteristic of the plurality of known characteristics to the audio data of the caller.