A hand tool assembly with magnetic securing device includes a body, a sleeve, a collar and two resilient pieces. The body has a first space and a first groove. The sleeve has a first passage, a second space, a third space, a first distance and a fourth space. A magnetic part is located in the second space. A clip is engaged with the first groove and movable within the third space. A collar is secured in the fourth space to restrict the clip to prevent the body from dropping form the sleeve. The collar has two resilient pieces received therein and each resilient piece has a third engaging portion which contacts the outer periphery of the body to provide friction when the sleeve is moved relative to the body. A driving piece is connected to the body and drives a bolt or a screw which is attracted by the magnetic part.