An exhaust emission control system of an internal combustion engine includes: a bypass passage (22) provided in an exhaust passage (14) of the engine and arranged to bypass a main passage (16) as a part of the exhaust passage, a NOx adsorbent (28) provided in the bypass passage and adapted to adsorb at least NOx as one of components contained in exhaust gas, a channel switching device (30) that switches a channel of the exhaust gas between the main passage and the bypass passage, an adsorption control device (50) that controls the channel switching device, based on operating conditions of the engine, so as to cause the exhaust gas to flow through the bypass passage, and an adsorbing capability determining device (50) that determines, when the adsorbing capability of the NOx adsorbent degrades, whether the degradation in the adsorbing capability is a surmountable degradation from which the NOx adsorbent can recover, or an insurmountable degradation from which the NOx adsorbent cannot recover.