Distributed machine learning on heterogeneous data platforms
Abstract:
A distributed machine learning framework implemented with heterogeneous data platforms reduces data copying and exploits memory/computation resources of the different data platforms. A configuration component includes information to set up the system. A persistency component manages storage of data and a model trained by machine learning. A proxy repository includes predefined proxies for communication between heterogeneous data platform nodes and execution of the machine learning procedure. A machine learning execution component comprises three layers. A bottom work node layer within the data platform performs computations of the machine learning procedure. A middle server node layer comprising one server node per data platform, communicates with the work nodes to coordinate jobs on that data platform. An upper layer comprises a central server node communicating with server nodes and coordinating jobs of the different platforms. The system can extend to additional external data platforms and external machine learning libraries with predefined proxies.
Public/Granted literature
Information query
Patent Agency Ranking
0/0