A shrink-fit tool unit increases the transmission force of rotary power and also improves workability when inserting the rotary tool into the tool holder. A tool holder and a rotary tool to be used for the shrink-fit tool unit are also disclosed. Since a first engaging part and a second engaging part are configured to engage with each other simply by inserting a cylindrical terminal area of a rotary tool, typically, a thread forming tap, into a chuck of the tool holder, even when the machining load receiving from a work piece act both in forward and reverse directions relative to the circumferential direction, it is possible to increase the transmission of rotary power.