The invention discloses an audio signal processing apparatus and method and a monitoring system. The audio signal processing apparatus includes: a window dividing unit for sequentially reading an inputted audio signal using a sliding window; an energy calculating unit for calculating energy of each frame of the audio signal in each window; a segmenting unit for segmenting, according to distribution of the energy of each frame of the audio signal in each window, the audio signal in the window into multiple segments, such that each segment includes successive frames with approximate energies; a classifying unit for classifying the audio signal in each segment using at least one sound model; and a recognizing unit for recognizing a sound class of the audio signal in each segment according to a result of the classifying by the classifying unit.