Systems and methods of parallel and distributed processing of datasets for model approximation
Abstract:
A system including: at least one processor; and at least one memory having stored thereon computer program code that, when executed by the at least one processor, controls the system to: receive a data model identification and a dataset; in response to determining that the data model does not contain a hierarchical structure, perform expectation propagation on the dataset to approximate the data model with a hierarchical structure; divide the dataset into a plurality of channels; for each of the plurality of channels: divide the data into a plurality of microbatches; process each microbatch of the plurality of microbatches through parallel iterators; and process the output of the parallel iterators through single-instruction multiple-data (SIMD) layers; and asynchronously merge results of the SIMD layers.
Information query
Patent Agency Ranking
0/0