A distributed machine learning engine is proposed that allows for optimization and parallel execution of the machine learning tasks. The system allows for a graphical representation of the underlying parallel execution and allows the user the ability to select additional execution configurations that will allow the system to either take advantage of processing capability or to limit the available computing power. The engine is able to run from a single machine to a heterogeneous cloud of computing devices. The engine is capable of being aware of the machine learning task, its parallel execution constraints and the underlying heterogeneous infrastructure to allow for optimal execution based on speed or reduced execution to comply with other constraints such as allowable time, costs, or other miscellaneous parameters.