An inattentiveness determination device includes a photographed image acquisition means configured to acquire a photographed image of a driver driving a moving object, a first inattentiveness determination means which detects a gaze direction of the driver according to the photographed image, and determine whether or not the driver is in an inattentiveness state on a basis of a predetermined gaze range based on a moving direction of the moving object and the gaze direction, and a second inattentiveness determination means configured to, if the gaze direction cannot be detected, determine whether or not the driver is in the inattentiveness state on a basis of a face direction of the driver and a predetermined range of the face direction based on the moving direction of the moving object.