Changes between Version 2 and Version 3 of waue/2009/1119


Ignore:
Timestamp:
Nov 19, 2009, 4:46:59 PM (15 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • waue/2009/1119

    v2 v3  
    1010
    1111 = 1.  =
    12  == A Dynamic MapReduce Scheduler for Heterogeneous Workloads ==
     12 * A Dynamic MapReduce Scheduler for Heterogeneous Workloads
     13
     14Abstract—MapReduce is an important programming model for building data centers containing ten of thousands of nodes. In a
     15practical data center of that scale, it is a common case that I/Obound
     16jobs and CPU-bound jobs, which demand different
     17resources, run simultaneously in the same cluster. In the
     18MapReduce framework, parallelization of these two kinds of job
     19has not been concerned. In this paper, we give a new view of the
     20MapReduce model, and classify the MapReduce workloads into
     21three categories based on their CPU and I/O utilization. With
     22workload classification, we design a new dynamic MapReduce
     23workload predict mechanism, MR-Predict, which detects the
     24workload type on the fly. We propose a Triple-Queue Scheduler
     25based on the MR-Predict mechanism. The Triple-Queue
     26scheduler could improve the usage of both CPU and disk I/O
     27resources under heterogeneous workloads. And it could improve
     28the Hadoop throughput by about 30% under heterogeneous
     29workloads.
     30