A vague assisted association analysis approach to repair Bigdata impurities

Abstract

Big data consist of huge amount of data. There are many of challenges when we accessing Big data. One of such major challenge is identified in terms of dataset impurities. These impurities are visible sometimes in terms of missing or incomplete information. But sometimes this kind of impurities is hidden in terms of non-valuable attribute or unnecessary information. In this paper, a two layered work is defined to improve the dataset integrity. In first phase, the analysis over the dataset is performed and later on the impurities are removed. Once the problems are identified, the particular attribute or the tuple are removed from the dataset. To verify the dataset integrity, th eassociation rules are generated.