Several embodiments of memory devices and systems with removable storage are disclosed herein. In one embodiment, a non-volatile dual in-line memory module (NVDIMM) includes a controller and a non-volatile memory slot configured to operatively connect a removable non-volatile memory device to the controller. The NVDIMM further comprises one or more volatile memories operatively connected to the controller. The controller is configured to backup content on the one or more volatile memories onto a removable non-volatile memory device operatively connected to the controller via the non-volatile memory slot. In some embodiments, the NVDIMM further comprises dedicated hardware configured to direct the controller to backup content on the one or more volatile memories onto a removable non-volatile memory device operatively connected to the controller via the non-volatile memory slot.