1 Introduction

In my most recent publications, I have dealt extensively with individual topics in the field of regression analysis. This post should serve as a summary of the topics covered.

2 Roadmap for Regression Analysis

Here are the links to the individual topics.

Data pre-processing:

• Dealing with outliers
• Handling Missing Values
• Feature Encoding
• Feature Scaling

Feature Selection:

Filter methods:

• Dealing with highly correlated features
• Dealing with constant features
• Dealing with duplicate features

Wrapper methods:

• SelectKBest
• Forward Feature Selection
• Backward Elimination
• Recursive Feature Elimination (RFE)

Embedded methods:

• Ridge
• Lasso
• Elastic Net

3 Different types of regression models

• Linear Regression
• Multiple Regression
• Non-Linear Regression
• Polynomial Regression

4 Further Regression Algorithms

• Decision Tree Regression
• Support Vector Machines Regression
• Stochastic Gradient Descent (SGD) Regression
• KNN Regression
• Bagging Regressor
• Bagging Regressor with Decision Tree Reg as base_estimator
• Random Forest Regressor
• AdaBoost Regressor
• AdaBoost Regressor with Decision Tree Reg as base_estimator
• Gradient Boosting Regressor
• XGBoost Regressor
• Stacking Regressor

5 Regression with Neural Networks

Of course, in addition to traditional regression algorithms, neural networks can be used to solve regression problems.

Here again are the links to the respective publications:

• Multi-layer Perceptron Regressor (MLPRegressor)
• Artificial Neural Network for Regression Analysis

6 Metrics for Regression Analysis

• Metrics for Regression Analysis

7 Conclusion

The methods mentioned in the listed posts show how to improve regression models.

The focus of the upcoming publications will be on algorithms for classification problems.