There are provided with an IP telephone system having both convenience of the softphone and durability of the hardphone, and a speech processing peripheral device (10) for the IP telephone system. The speech processing peripheral device as a peripheral device of an information processing device (20) is connected to the information processing device (20) for executing a call control program and functioning as a call controller. This speech processing peripheral device (10) includes a host connection means for sending/receiving an audio signal to/from the information processing device (20) and a speech processing means for executing speech processing in IP telephone communication. The IP telephone system according to the present invention includes the information processing device (20) for executing the call control program and functioning as the call controller and the speech processing peripheral device (10) according to the present invention, connected to the information processing device (20).