Resource management in MapReduce architecture and architectural system
Abstract:
Resource management in MapReduce architecture and architectural system. The present invention provides an apparatus for resource management in the MapReduce architecture including a memory, a processor communicatively coupled to the memory, and a module for resource management in the MapReduce architecture configured to carry out the steps of a method. The method includes the steps of: determining a ratio r of an input data amount of a Map task, an output data amount of the Map task, and an average size R of a record in Map output results; determining a memory size Memory_size that can be allocated to the Map task corresponding to a Map slot; determining an input split size appropriate for the Map task according to the determined r, R and Memory_size; and allocating an input split with the input split size in a MapReduce job to be processed to the Map task occupying the Map slot.
Information query
Patent Agency Ranking
0/0