Big Data and decision-making processes in the 5 V dimension

Abstract

The high use of computers has created large volumes of data that cannot be managed by traditional software and hardware. Take the case of large companies like Microsoft and Google, which must store, manage and manipulate billions of pieces of data. This perplexity in managing these large volumes of data gave birth to the term Big Data. The potentially infinite amounts of data, as well as the constraints that derive from it, pose many problems in terms of storage and processing of these very large data sets in terms of time and calculation using dedicated platforms such as Hadoop, which is one of the best platforms for Big Data and which is based on the Map Reduce paradigm.

Technological evolution in the 21st century; from day to day the data evolves in time and space, from space to space we talk about Big data and the cluster in the decision-making process in the 5 V dimension, this is the very important research topic that interests many researchers in the unsupervised classification of data, and it is a research axis no less important than the 5 V itself, several indices of data validation have been proposed in the literature review either for external acceptance or internal.

External acceptance requires a priori knowledge of the optimal real partition and is based on the comparison of the big data obtained by a supervised learning algorithm for any KNN classification with the known optimal partition. The most well-known error indices are: purity, measure, entropy, synaptic coefficient, etc.

In this work, we proposed a parallel and distributed model to solve the problem of scaling external validation indices of Big Data, and more precisely the "synaptic coefficient" index. We used the Hadoop platform, and more precisely the Map Reduce paradigm, for the implementation of the proposed model. The results obtained show the validity of this model.

Keywords: Big Data, supervised learning, decision process, clustering, Hadoop, Map Reduce, 5 V dimension, cluster and KNN classification, synaptic coefficient.

Received Date: August 21, 2025              

Accepted Date: September 13, 2025            

Published Date: October 01, 2025

Available Online at https://www.ijsrisjournal.com/index.php/ojsfiles/article/view/491