A base station (BS) for handling bandwidth parts (BWPs) comprises at least one storage device storing instructions of communicating with a communication device via a first DL BWP and a first UL BWP in a cell; transmitting a radio resource control (RRC) message in the first DL BWP to the communication device, wherein the RRC message configures at least one of a second DL BWP and a second UL BWP in the cell to the communication device; receiving a RRC response message from the communication device in the first UL BWP; receiving a RA preamble from the communication device in the first UL BWP or the second UL BWP; transmitting a RA response (RAR) to the communication device in the first DL BWP or the second DL BWP; and communicating with the communication device via the at least one of the second DL BWP and the second UL BWP.