An ultracapacitor or hybrid capacitor includes an electrically non-conductive rigid or semi-rigid porous honeycomb separator structure having cells extending along a common direction and supporting current collector structure(s) thereon. The current collector structure may be porous and extend continuously on all inner surfaces of a cell of the honeycomb structure, or may extend along the common direction on separate portions of the inner surfaces of a cell. The material may desirably be an oxide or non-oxide ceramic, such as cordierite, silicon nitride, aluminum titanate, alumina, zircon, glass, or glass-ceramic.