When a serving base station encounters a trigger for handover of a UE to a target base station, the serving base station will responsively predict whether, before the serving base station would direct the UE to handover to the target base station, an inactivity timer for the UE will expire, and the serving base station will then control the handover based on that prediction. For instance, responsive to the predicting being that the inactivity timer will expire before the serving base station would send a handover command message to the UE, the serving base station could forgo engaging in handover-preparation signaling with the target base station for the handover, instead allowing the UE to transition to idle mode upon expiration of the timer, at which point the UE could then engage in idle-mode handover to the target base station.