Methods and apparatus for conducting test promotions in a highly scalable and cost-effective manner using controlled experiment methodology are disclosed. Test promotions of interest are presented to visitors of a website in a controlled experiment manner wherein each page presents one test promotion of interest among other promotions. The other promotions presented in the same page are kept constant in the pages. The visitors' responses with respect to the test promotions of interest are then recorded and analyzed to determine the performance of each presented test promotion of interest.