Techniques are disclosed for modifying an application-level configuration of a cluster of virtual application servers in response to an automatic provisioning of server instances to or from the cluster. An auto-scaling service receives an indication that one or more metrics associated with a cluster in a service provider environment triggers a specified scaling condition. The cluster includes application server instances executing one or more services provided by an application. The auto-scaling service provisions an additional application server instance in response to the indication. One or more services provided by the application are installed on the additional application server instance. A configuration engine obtains one or more scripts for configuring the additional application server instance to execute the services as part of the cluster. The scripts are executed on the application server instances to include the additional application server instance to the cluster.