Provided is an X-ray signal processor and an X-ray spectrometer that are configured to measure X-rays and, at the same time, accurately detect, with a simple method, the degree of performance degradation of a semiconductor detector. The X-ray signal processor includes: a semiconductor detector configured to generate a charge corresponding to energy of detected X-rays; a preamplifier configured to output a ramp voltage signal corresponding to the generated charge; a counter configured to count the X-rays for each voltage change amount due to the charge based on the ramp voltage signal; and a judgment part configured to determine whether the semiconductor detector has been degraded based on a first voltage change evaluation value corresponding to a total sum of products of the voltage change amount and an occurrence frequency thereof, and a second voltage change evaluation value corresponding to an increase amount of the ramp voltage signal.