The present disclosure provides an optical touch system and an object analyzation method thereof. The optical touch system includes a panel and an image sensing apparatus installed thereon. The object analyzation method includes the following steps: capturing a first image across a touch surface of the panel with an image sensing apparatus, wherein the first image has an object image formed corresponding to the position of a pointer on the touch surface; defining an image window that corresponds to the image position of the object image in the first image captured; determining whether the pointer is touching the touch surface or hovering over the touch surface by analyzing the brightness difference among the plurality of pixels in the image window.