Preliminary Results: Modeling Relation Between Total Execution Time of MapReduce Applications and Number of Mappers/Reducers

Select |




Print


Babaii Rizvandi, Nikzad; Zomaya, Albert; Boloori, Ali Javadzadeh; Taheri, Javid


2011-08-15


Report


Sydney, Australia


In this paper, we propose an analytical method to model the dependency between configuration parameters and total execution time of Map-Reduce applications. Our approach has three key phases: profiling, modeling, and prediction. In profiling, an application is run several times with different sets of MapReduce configuration parameters to profile the execution time of the application on a given platform. Then in modeling, the relation between these parameters and total execution time is modeled by multivariate linear regression. Among the possible configuration parameters, two main parameters have been used in this study: the number of Mappers, and the number of Reducers. For evaluation, two standard applications (WordCount, and Exim Mainlog parsing) are utilized to evaluate our technique on a 4-node MapReduce platform.


MapReduce, Configuration parameters, total execution time, multivariate linear regression


1833-9646-5181


nicta:5181


Babaii Rizvandi, Nikzad; Zomaya, Albert; Boloori, Ali Javadzadeh; Taheri, Javid. Preliminary Results: Modeling Relation Between Total Execution Time of MapReduce Applications and Number of Mappers/Reducers. 2011-08-15. http://hdl.handle.net/102.100.100/103404?index=1



Loading citation data...

Citation counts
(Requires subscription to view)