A system for controlling a compressor may include an engine controller that controls a fuel injection amount corresponding to an engine load and an opening amount of a throttle by reflecting a required torque required for an air conditioner (A/C), an operation information detector for detecting operation information according to driving state of the vehicle, a compressor that generates pressure during operation of the A/C, an air conditioner relay which is turned on when the air conditioner operates and is turned off when the A/C is stopped, and a controller which determines an engine negative pressure of an intake manifold, and when the cooling water temperature is lower than the predetermined temperature and the intake manifold pressure is lower than the first threshold value, a cold-start intake manifold negative pressure insufficient event is generated to reduce the A/C duty in accordance with the entry into a negative pressure recovery mode.