A micro electrical-mechanical systems (MEMS) device INCLUDES a MEMS substrate and at least one MEMS structure on the MEMS substrate. In addition, there is at least one battery cell on the MEMS substrate coupled to the at least one MEMS structure. The at least one battery cell includes a support fin extending vertically upward from the MEMS substrate and a first electrode layer on the support fin. In addition, there is an electrolyte layer on the cathode layer, and a second electrode layer on the electrolyte layer. The support fin may have a height greater than a width. The first electrode layer may have a processing temperature associated therewith that exceeds a stability temperature associated with the second electrode layer.