A technique provides secure access to a set of credentials within a data storage system. The technique involves obtaining a unique identifier (e.g., a hostname which is unique to the system) and a set of stable values (e.g., machine-generated codes which are random to users of the system); and, in response to a storage request from a client application, storing a set of credentials of the client application within a data security mechanism of the data storage system. The set of credentials is in encrypted form when stored within the data security mechanism of the data storage system. The technique further involves configuring the data security mechanism of the data storage system to provide the set of credentials in non-encrypted form in response to new fingerprints matching a system fingerprint which is formed at least in part from the unique identifier and the set of stable values.