A pin and sleeve device including a contact carrier with captive set screws is disclosed. In some embodiments, the pin and sleeve device includes a housing and a contact carrier. The contact carrier may include a plurality of contact openings each receiving a terminal, such as a pin or a contact sleeve. The contact carrier may include a tool opening or slot extending from an outer surface or perimeter thereof. The tool opening or slot being in communication with one of the contact openings and/or terminals. A set screw may be positioned within the contact openings, the set screw positioned directly adjacent a retainment wall defining an inner portion of the contact openings. The contact carrier may therefore reduce the likelihood that the set screw may be lost or dropped, as the retainment wall is configured to prevent the set screw from passing through or exiting the tool opening or slot.