A method of starting a polymer electrolyte membrane fuel cell (PEMFC) stack by rapidly increasing its temperature. The PEMFC stack includes: a first flow line connected to cooling plates; a second flow line connected to the cooling plates; a coolant reservoir; a heat exchanger; a by-pass line; a heating element; a first valve installed between the first flow line and the heat exchanger; and a second valve that selectively connects the coolant reservoir, the second flow line, and the by-pass line. The method of starting a PEMFC stack includes: closing the first valve and controlling the second valve so that the second flow line and the by-pass line are connected to each other, and the coolant in the coolant reservoir is not connected to the second flow line and the by-pass line; and heating the coolant in the by-pass line.