¿Qué es la regresión de ridge?
Cresta Regresión, also known as Tikhonov regularization, is a type of regresión lineal that includes a regularization term to prevent overfitting. This technique is particularly useful when dealing with multicollinearity, where independent variables are highly correlated.
In standard linear regression, the goal is to minimize the sum of the squared differences between the observed and predicted values. However, when the model is too complex or when there are many predictors, it can lead to overfitting, where the model performs well on datos de entrenamiento pero funciona mal con datos no vistos.
La Regresión de Cresta aborda este problema añadiendo un término de penalización a la función de pérdida, which is proportional to the square of the magnitude of the coefficients. The modified loss function can be expressed as:
Pérdida = Suma de residuos al cuadrado + λ * (Suma de cuadrados de los coeficientes)
Here, λ (lambda) is a tuning parameter that controls the strength of the penalty. A larger value of λ increases the penalty on the coefficients, leading to smaller coefficient values. This helps in making the model more generalizable by reducing its complejidad.
Ridge Regression is particularly effective when you have many predictors and a smaller number of observations, often leading to a model that performs better on test data compared to regular linear regression. It is important to note that while Ridge Regression can shrink coefficients, it does not perform variable selection (i.e., it does not set any coefficients exactly to zero). This is where techniques like Regresión Lasso, which can perform variable selection, come in handy.
En general, la Regresión de Cresta es una herramienta poderosa en el aprendizaje automático toolbox, helping to create robust predictive models by balancing the trade-off between fitting the training data and maintaining model simplicity.