A coupling device includes a body having an end for coupling with a handle. The body includes a compartment having a receiving space for receiving an object such as a bit. The receiving space includes a slanted face extending longitudinally outward and radially inward. A follower is slidably mounted in the compartment and includes an outer end having a restraining hole. A positioning member extends from a radially outer side through the restraining hole to a radially inner side of the follower and is slidable along the slanted face. The follower is biased outward by an elastic element toward the receiving space when the object is inserted into the receiving space and moves the follower longitudinally inward. A radial clamping force is imparted from the slanted face through the positioning member to securely clamp the object in the receiving space.