An infant safety seat comprises a seat shell, a handle and at least a latch element mounted between the seat shell and the handle. Two sides of the seat shell are provided with first coupling elements, each of which includes multiple locking grooves. Two sides of the handle has second coupling elements that are pivotally connected with the first coupling elements, each of the second coupling elements having an interior defining a guide slot. The latch element is movably assembled in the guide slot and includes at least a first stud, the latch element being operable to switch between a locked state where the first stud is engaged with one of the locking grooves to hold the handle in place, and an unlocked state where the first stud is disengaged from the locking grooves to allow rotation of the handle relative to the seat shell.