"EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"
Abstract & Details
Research Area
Computer Engineering
Keywords
BigData
MapReduce
Incremental Processing
MRBG
Abstract
Map Reduce programming model is widely used for large scale and one-time data-intensive distributed computing, but lacks flexibility and efficiency of processing small incremental data. Incremental Mapreduce framework is proposed in this paper for incrementally processing new data of a large data set, which takes state as implicit input and combines it with new data. Map tasks are created according to new splits instead of entire splits while reduce tasks fetch their inputs including the state and the intermediate results of new map tasks from Preserved and latest generated result. The preserved states really producing the promising result and significantly reduce the run time for refreshing big data mining results compared to re-calculating on both simple and multi stage MapReduce. The intermediate states are saved in the form of kv-pair level data flow and data dependence in a MapReduce computation as a bipartitegraph, called MRBGraph. A MRBG-Store is designed to preserve the fine-grain states in the MRBGraph and support efficient queries to retrieve fine-grain states for incremental processing.
License
This work is licensed under a Creative
Commons
Attribution-ShareAlike 4.0 International License.
Author Information
| # | Name | Institute / Affiliation |
|---|---|---|
| 1 | Mr.Nilesh Vasant Kurhade | SPIET.Otur(Pune) |
| 2 | Prof.Kahate Sandip. | SPCOE,Otur(Pune) |
How to Cite
Use the following formats to cite this article in your research.
APA Style
Kurhade, Mr.Nilesh Vasant & Sandip., Prof.Kahate (2016). "EXMR; An Enhanced way for Incremental Data Processing based on MapReduce". International Journal of Advance Research and Innovative Ideas In Education, 2(1), 135-140.
MLA Style
Kurhade, Mr.Nilesh Vasant, and Prof.Kahate Sandip.. ""EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"." International Journal of Advance Research and Innovative Ideas In Education, vol. 2, no. 1, 2016, pp. 135-140.
IEEE Style
Mr.Nilesh Vasant Kurhade and Prof.Kahate Sandip., ""EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"," International Journal of Advance Research and Innovative Ideas In Education, vol. 2, no. 1, pp. 135-140, 2016.
Vancouver Style
Kurhade Mr.Nilesh Vasant, Sandip. Prof.Kahate. "EXMR; An Enhanced way for Incremental Data Processing based on MapReduce". International Journal of Advance Research and Innovative Ideas In Education. 2016;2(1):135-140.
Harvard Style
Kurhade, Mr.Nilesh Vasant & Sandip., Prof.Kahate (2016) '"EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"', International Journal of Advance Research and Innovative Ideas In Education, 2(1), pp. 135-140.
Chicago Style
Kurhade, Mr.Nilesh Vasant and Prof.Kahate Sandip.. ""EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"." International Journal of Advance Research and Innovative Ideas In Education 2, no. 1 (2016): 135-140.
Turabian Style
Kurhade, Mr.Nilesh Vasant and Prof.Kahate Sandip.. ""EXMR; An Enhanced way for Incremental Data Processing based on MapReduce"." International Journal of Advance Research and Innovative Ideas In Education 2, no. 1 (2016): 135-140.
Related Research
A Brief Review of Performance Evaluation of Global and Local Image Thresholding Based on Computational Efficiency
Download PDF
Performance Evaluation of Global and Local Image Thresholding Based on Computational Efficiency
Download PDF
Enhancing Handwritten Alpha-Numeral Recognition Using GAN-Based Synthetic Data Generation
Download PDF
Comprehensive Review of Existing Chatbot Systems for Career Assistance, Resume Support, and ATS-Aware Guidance
Download PDF
Development of an AI-Powered Multimodal Web Assistant with Intelligent Resume Building and ATS Enhancement
Download PDF
A Deep Learning-Based Framework for Mood-Oriented Music Recommendation Using Facial Expression Analysis
Download PDF