The present invention provides a pin retainer comprising a base, and a retaining structure coupled to the base. The base has a through slot, two through-holes respectively formed at two opposite sides of an opening of the through-slot formed on a surface of the base for respectively accommodating guide pins, and first engaging structures respectively formed on two lateral surfaces of the base. The retaining structure has a pair of lateral plates slidably coupled to the two lateral surfaces. Each lateral plate has second engaging structure corresponding to the first engaging structure, and a positioning plate having a holding structure. The second engaging structure is combined with or released from the first engaging structure depending on the positions of the retaining structure. Alternatively, the present invention also presents an optical connector using the pin retainer.