A pyroelectric ceramic composition contains a compound represented by (Pb1−xCax)(1+a){(Ni1/3Nb2/3)yTi(1−y)}O3 (wherein x, y, and a satisfy 0.20≦x≦0.27, 0.01≦y≦0.06, and 0.001≦a≦0.02, respectively) as a main component and 0.3 to 2.5 mol of Mn per 100 mol of the main component. The amount of segregates containing Ni, Ti, and Mn is 1.0 vol % or less (including 0 vol %) of a fired sinter. A pyroelectric ceramic composition that has an adequately low insulation resistance and a high Curie temperature while achieving satisfactory pyroelectric properties is realized. Since the composition has a high Curie temperature, a thin, small pyroelectric element that withstands a reflow process can be obtained. Since the composition has a low insulation resistance, an infrared detector incorporating a pyroelectric element composed of the composition does not require a load resistance to be provided in parallel to the pyroelectric element and size reduction is possible.