Provided is a tool holding structure capable of effectively preventing a cutting tool from coming off a chuck portion. An outer fitting portion A is formed in an outer face of a shank portion 2 of a cutting tool T and an inner fitting portion B is formed in an inner face of a chuck portion 12 of a tool holder. A fitting member 20 is provided to be fitted to the outer fitting portion A and the inner fitting portion B respectively. The fitting member 20 includes a protruding portion 22 which protrudes from an outer circumference of the shank portion 2 when fitted to the outer fitting portion A. A guide groove 18 is formed for guiding the protruding portion 22 of the fitting member 20 from an open side of an inner space 13 to the inner fitting portion B. The inner fitting portion B includes a rotation preventing face for preventing relative rotation between a holder body 10 and the cutting tool T in association with contact thereof with the protruding portion 22 and a slip-out preventing face for preventing slip-out of the cutting tool T in association with contact thereof with the protruding portion 22.