Comparative Study of Missing Value Imputation Techniques on E-Commerce Product Ratings

Abstract

Missing data is a common occurrence in practically all studies, and it adds a layer of ambiguity to data interpretation. Missing values in a dataset mean loss of important information. It is one of the most common data quality issues. Missing values are values that are not present in the data set. These are usually written as NAN’s, blanks, or any other placeholders. Missing values create imbalanced observations, biased estimates and sometimes lead to misleading results. The majority of real-world datasets have missing values. As a result, to deliver an efficient and valid analysis and the solutions should be taken into account appropriately. By filling in the missing values a complete dataset can be created and the challenge of dealing with complex patterns of missingness can be avoided. Missing values can be of both continuous and categorical types. To get more precise results, a variety of techniques to fill out missing values can be used. In the present study, nine different imputation methods: Simple Imputer, Last Observation Carried forward (LOCF), KNN Imputation (KNN), Hot Deck, Linear Regression, MissForest, Random Forest Regression, DataWig, and Multivariate Imputation by Chained Equation (MICE) were compared. The comparison was performed on Amazon real-time dataset based on three evaluation criteria: R- Squared (R²), Mean squared error (MSE), and Mean absolute error (MAE). As a result of the findings KNN had the best outcomes, while DataWig had the worst results for R- Squared (R²). The R-squared value ranges from 0-1. In terms of mean squared error (MSE) and mean absolute error (MAE), the Hot deck imputation approach fared best, whereas MissForest performed worst (MAE). The hot deck imputation method appears to be of interest and merits further investigation in practice.