An arm control system for a work vehicle includes a controller having a memory and a processor. The controller is configured to receive a signal indicative of a type of implement coupled to an arm of the work vehicle. The controller is also configured to enable movement of the arm between a lowered position and a raised position while the type of implement is an unrestricted-height implement. In addition, the controller is configured to enable movement of the arm between the lowered position and an intermediate position and to block upward movement of the arm beyond the intermediate position while the type of implement is a restricted-height implement.