A brightness sensing system includes an image pickup unit for taking an image of a target space and a calculation unit for calculating the brightness of an image pickup area based on the image taken by the image pickup unit. In the brightness sensing system, if the difference or the ratio between an image as a whole and a pixel block formed of one or more pixels is equal to or greater than a threshold in terms of brightness change between a previously-taken reference image and a currently-taken target image, the calculation unit excludes the pixel block from comparison candidates and compares the brightness of the target image with the brightness of the reference image based on the remaining pixels other than the pixel block.