A seat belt warning device includes a seat belt restraining and releasing a passenger seated on a seat for a vehicle, a load detecting device including two load sensors that are provided at a rear portion of the seat, a warning member warning a passenger to fasten the seat belt, and a controlling portion determining one of an unoccupied-seat recognition mode, a passenger seating recognition mode, and a luggage recognition mode. The controlling portion performs a first transition process in which the unoccupied-seat recognition mode is shifted to the luggage recognition mode and in which the luggage recognition mode is shifted to the unoccupied-seat recognition mode. The controlling portion performs a second transition process in which the unoccupied-seat recognition mode is shifted to the passenger seating recognition mode and in which the passenger seating recognition mode is shifted to the unoccupied-seat recognition mode.