A method and apparatus are disclosed from the perspective of a first UE (User Equipment) to indicate a network node for releasing SLRB (Sidelink Radio Bearer) or SL-DRB (Sidelink-Data Radio Bearer). In one embodiment, the method includes transmitting a first RRC (Radio Resource Control) message to a network node for requesting sidelink resource, wherein the first RRC message includes a PFI (PC5 QoS Flow Identifier) of a PC5 QoS (Quality of Service) flow in a first list of PC5 QoS flows. The method also includes receiving a second RRC message from the network node, wherein the second RRC message configures a SLRB and maps the PC5 QoS flow to the SLRB. The method further includes transmitting a third RRC message to the network node if the PC5 QoS flow is deactivated or released, wherein the PFI of the PC5 QoS flow is removed from a second list of PC5 QoS flows included in the third RRC message.