IMPROVING TWO-WAY JOIN PROCESSING USING BLOOM FILTER INTERSECTION IN MAP REDUCE

Computer Engineering

Big Data Analysis is at the heart of modern enterprise and scientific research. To analyze this very large-scale data, MapReduce has become an attractive model for this kind of data. However, this model is not designed to perform join operations with multiple entries. Although many studies on join algorithms like Bloomjoin in MapReduce have been explored, but a lot of non-joining data is still being generated and transmitted over the network. This research will provide us with a model to solve this problem by making an intersection filter based on the Bloom Filter to eliminate disjoint elements between two sets of input data. Two methods are proposed to model the filter. To apply the filter in a join operation, a MapReduce job will be adjusted consistently, attempting not to increase the associated costs. Therefore, thanks to the intersection filter, the operation of the joins minimizes the cost of disk I/O. Finally, the study carried out proved its effectiveness by comparing with the existing solutions on the costs of the joins of the two approaches.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.