There is provided herein a system and method for enabling a user of digital video editing software to automatically create an emotionally controlled soundtrack that is matched in overall emotion or mood to the scenes in the underlying video work. In the preferred arrangement, the user will be able to control the generation of the soundtrack by positioning emotion tags in the video work that correspond to the general mood of each scene. The subsequent soundtrack generation step will utilize these tags to prepare a musical accompaniment to the video work that generally matches its on-screen activities.