Methods, systems, and apparatus, including computer programs encoded on computer storage media, for unmanned aerial vehicle modular command priority determination and filtering system. One of the methods includes enabling control of the UAV by a first control source that provides modular commands to the UAV, each modular command being a command associated with performance of one or more actions by the UAV. Modular commands from a second control source requesting control of the UAV are received. The second control source is determined to be in control of the UAV based on priority information associated with each control source. Control of the UAV is enabled by the second control source, and modular commands are implemented.