In a cooling water circulation device for use in a NOx reduction system where a urea injector 3 is positioned higher than an engine 1, a three-way branching device 10 is arranged at a highest position in a passage of passing cooling water 2 from the engine 1 to the urea injector 3 and returning the same to the engine 1 to add an upward air vent 10a. The air vent 10a is connected through an air-venting connecting pipe 11 to a header tank 9 (water pouring tank) for the cooling water 2 arranged higher than the urea injector 3.