Provided are a computer program product, system, and method for a data set connection manager having a plurality of data sets to represent one data set. A request is processed to open a connection to a data set having members, wherein the connection is used to perform read and write requests to the members in the data set. In response to establishing the connection, establishing for the connection a primary data set having all the members; a secondary data set to which updated members in the primary data set are written; and a pending delete data set to pending delete members comprising members that are updated.