Systems and methods for retaining magnetic debris from drilling fluid include an exterior sleeve with an interior space receiving an interior holder. A first end of the exterior sleeve is connectible to drill string to place the interior space in fluid communication with the drill string, and a longitudinal passage between the interior holder and exterior sleeve defines a flow path for drilling fluid from the drill string. Magnetic materials along and/or around the interior holder provide a magnetic field extending into or across the longitudinal passage such that magnetic debris is removable by the magnetic field from the flow path of drilling fluid passing through the longitudinal passage.