A light emitting device is described. The light emitting device includes a substrate and a semiconductor structure. The semiconductor structure includes a light emitting layer disposed between an n-type region and a p-type region and has a first surface adjacent the substrate and a second surface opposite the first surface. The first surface of the semiconductor structure multiple cavities formed therein, which extend into at least one of the n-type region and the p-type region. The cavities are spaced apart and lined by a dielectric layer. At least a portion of the second surface is roughened to form multiple features spaced apart at a distance smaller than a distance between each of the cavities formed in the first surface to enhance extraction of light emitted from the light emitting layer. At least one contact is disposed between the first surface of the semiconductor structure and the substrate.