A light source device has: a light source unit including solid-state light sources emitting blue-color light and a condensing lens; a dichroic mirror polarization separating color light into first and second polarization components; a fluorescence emission plate excited with the first polarization component, emitting fluorescence of green and red components to enter the dichroic mirror; a first retardation plate converting the second polarization component to a circularly polarized light; and a reflection plate reflecting a light passed through the first retardation plate to reenter the same. The color lights from the fluorescence emission plate and the first retardation plate are combined at the dichroic mirror. A second retardation plate converts the polarization direction of the light from the light source unit, so as to control the ratio of p-polarization and s-polarization components. Lights from solid-state light sources are condensed efficiently and a high spectrum-utilization factor is obtained.