Información Mutua (MI) is a statistical measure that quantifies the amount of information obtained about one random variable through another random variable. It is particularly useful in fields like teoría de la información, statistics, and aprendizaje automático.
Matemáticamente, la Información Mutua entre dos variables aleatorias discretas X y Y se define como:
MI(X; Y) = ∑∑ P(x, y) log( P(x, y) / (P(x) P(y)) )
donde:
- P(x, y) is the distribución de probabilidad conjunta de X y Y.
- P(x) is the probabilidad marginal distribución de X.
- P(y) es la distribución de probabilidad marginal de Y.
La Información Mutua captura la reducción en uncertainty about one variable given knowledge of the other. If X and Y are independent, MI(X; Y) equals zero, indicating no shared information. Conversely, a higher MI value indicates a stronger relationship and greater amount of shared information between the two variables.
In practical applications, MI is widely used in feature selection, where it helps identify the most informative features that contribute to a predictive model. It is also employed in clustering, registro de imágenes, and analyzing the dependencies between random variables in complex systems.