A tool for driving threaded fasteners includes a tool bit extending along a longitudinal axis and including a rear shank of polygonal cross section, a front working end, and an intermediate portion having a front shoulder, a rear shoulder, and a reduced diameter annular groove. A floating magnet sleeve receivable over the tool bit includes a tubular body, retention member projecting radially inward from a rear end of the body, and a magnet disposed at a front end of the body. The sleeve is axially moveable by a float distance between rearward and forward positions such that the magnet can engage a fastener head coupled to the working end as the sleeve approaches the forward position. The float distance is less than a tip distance between a front end of the magnet and a front tip of the working end when the sleeve is in the rearward position.