A thermal energy recovery device includes: a circulation line having an evaporator, an expander, a condenser, and a pump; a power recovery machine; a first on-off valve; a thermal energy introduction line configured to introduce a gas phase working medium into a post-expansion space; a second on-off valve; and a control unit. Until an evaporation condition that a liquid phase working medium accumulated in the post-expansion space has reached an amount equal to or smaller than a reference amount is met, the control unit closes the first on-off valve and opens the second on-off valve, and drives the pump in a state where the expander is stopped, and when the evaporation condition is met, the control unit opens the first on-off valve and closes the second on-off valve, and drives the expander.