A hydrocarbon resource recovery system may include an RF source, and an RF antenna assembly coupled to the RF source and within a wellbore in a subterranean formation for hydrocarbon resource recovery. The RF antenna assembly may include first and second tubular conductors, a dielectric isolator, and first and second electrical contact sleeves respectively coupled between the first and second tubular conductors and the dielectric isolator so that the first and second tubular conductors define a dipole antenna. The RF antenna assembly may include a thermal expansion accommodation device configured to provide a sliding arrangement between the second tubular conductor and the second electrical contact sleeve when a compressive force therebetween exceeds a threshold.