COMPARISM OF THE PENALIZED REGRESSION TECHNIQUES WITH CLASSICAL LEAST SQURES IN MINIMIZING THE EFFECT OF MULTICOLLINEARITY
CHAPTER ONE
INTRODUCTION
1.1 Background of the Study
In order to reduce possible biasness,large number of predictor variables was introduced in a model and that lead to a serious concern of multicollinearity among the predictor variables in multiple linear regressions, variable selection is an important issue.(Mathew and Yahaya, 2015)
Multicollinearity and high dimensionality are two problems and computational issue that bring challenges to regression analysis. To deal with these challenges, variables selection and shrinkage estimation are becoming important and useful. The traditional approach of automatic selection (such as forward selection, backward elimination and stepwise selection) and best subset selection are computationally expensive and may not necessarily produce the best model.
Multicollinearity problem is being dealt with by Penalized least square (PLS) method by putting some constraints on the values of the parameters estimated. The aftermath is that the entries of the variancecovariance matrix are significantly reduced.When multicollinearity exist that predictor’s variables that are highly correlated form some groups. One of the waycollinearity problem can be dealt with is to remove one or more of the predictor variables within the same group, by making decision which among the group variables is to be eliminated tend to be difficult and complicated. Theaftermath of multicollinearityis that the parameter estimator and their variance or standard error tends to be large and prediction may be very inaccurate.
In a situation where there exist correlated data or data where the number of predictors is much larger than the sample size, penalized regression methods have beenintroduced to deal with this challenge, because they produce more stable results, penalized regression methods do not clearly select the variables; instead they minimize the Regression Sum of Square by using a penalty on the size of the regression coefficients. This penalty causes the regression coefficients to shrink toward zeroand thismay result in biased estimates through these regression coefficient estimates will have smaller variance. This can improve the prediction accuracy because of the smaller mean squared error (Hastie et al., 2009). This is why penalized regression methods are also known as shrinkage or regularization methods. Some regression coefficients are set to zero exactly if the shrinkage is large enough,thus, penalized regression methods perform variable selection and coefficient estimation simultaneously. The Least Absolute Shrinkage Selection Operator (LASSO) enables selection such that only the important variable stays in the model (Szymeezak,et al., 2009).
DOWNLOAD COMPLETE PROJECT MATERIALS
COMPARISM OF THE PENALIZED REGRESSION TECHNIQUES WITH CLASSICAL LEAST SQURES IN MINIMIZING THE EFFECT OF MULTICOLLINEARITY
