An object detecting system, which detects an object by using a hierarchical pyramid structure includes: a pyramid image generator configured to receive a plurality of input images respectively corresponding to a plurality of time points and generate a plurality of pyramid images corresponding to each of the plurality of input images; an object extractor configured to generate a plurality of pieces of object data by extracting at least one object from the plurality of pyramid images; and a buffer storing the plurality of pieces of object data on an object basis.