Delivery systems and compositions comprised of a biodegradable polyorthoester polymer, an aprotic solvent, and a drug are described. The solvent is selected to modulate release of drug from the composition, where, in some embodiments, the solvent is rapidly released after administration and provides a corresponding rapid rate of drug release. Alternatively, in other embodiments, the solvent is slowly released from the composition after its administration, and provides a correspondingly slow rate of drug release.