An ultrasound probe assembly includes an ultrasound probe, an ultrasound cap, a solid hydrogel component, and a cover. The ultrasound probe can include a probe head and an orientation indicator. The probe head includes an acoustic surface. The ultrasound cap can include an opening defined by an opening perimeter, and a base configured to support a removable needle guide. The base can be aligned with the orientation indicator. The solid hydrogel component can include a probe-contacting side contacting the acoustic surface, a skin-contacting side extending through the opening, and an outer perimeter larger than the opening perimeter. The cover can be configured to cover the hydrogel component and at least a portion of the ultrasound cap.