A visual prosthesis for restoring color vision to patients with lost or impaired visual function. The prosthesis creates the perception of a color image by electrically stimulating visual neurons in patterns consistent with the electrically patterns naturally produced by the retina in response to a color image.