Provided are a computer program product, system, and method for determining priorities for cached objects to order the transfer of modifications of cached objects based on measured network bandwidth. Objects are copied from a primary site to a secondary site to cache at the secondary site. The primary site includes a primary server and primary storage and the secondary site includes a secondary server and a secondary storage. Priorities are received from the secondary server for the objects at the secondary site based on determinations made by the secondary server with respect to the objects cached at the secondary storage. A determination is made of modifications to the objects at the primary storage that are cached at the secondary storage. The received priorities for the objects from the secondary server are used to control a transfer of the determined modifications to the objects to the secondary server.