A method and apparatus are disclosed from the perspective of a first UE (User Equipment) to request SLRB (Sidelink Radio Bearer) configuration for a unicast link with a second UE. In one embodiment, the method includes the first UE receiving a first message from the second UE, wherein the first message includes a first SLRB configuration for the unicast link. The method further includes the first UE transmitting a second message to a network node to request a second SLRB configuration for the unicast link when the first message is received or a successful transmission of a complete message associated with the first message to the second UE is confirmed.