Disclosed is an internally illuminated bioreactor, and related algae production methods, that employ integrated in-water grow light assemblies configured to manage the heat generated by lighting elements, such as light emitting diodes (“LEDs”) on the in-water grow lights. The bioreactor includes an outer shell and one or more in-water grow light fixtures positioned within the outer shell that are positioned around the perimeter of a hollow, internal tube. The lighting elements and internal tube are themselves contained within a preferably clear, exterior tube of the light fixture that allows light generated by the lighting elements to pass through to the algae culture inside of the growth chamber. A heat management system is provided for cooling the light fixture using forced directed through the hollow, internal tube from the top to the bottom of the tube, out from outlets at the bottom of the internal tube, and upward in the fixture through buoyancy of the warmed air, and thus without additional mechanical air handling devices. As the air moves upward between the lighting elements and the exterior tube, it draws additional heat away from the lighting elements. The warmed air is ultimately exhausted from the top of the lighting fixture. Each lighting fixture preferably also includes a cleaning system that enables the automated cleaning of the outer surface of the exterior tube of the lighting fixture, thus preventing newly formed algae from collecting on the lighting fixture and ensuring a continuous flow of light from the fixture into the algae culture throughout algae production.