Information mutuelle (IM) is a statistical measure that quantifies the amount of information obtained about one random variable through another random variable. It is particularly useful in fields like théorie de l'information, statistics, and apprentissage automatique.
Mathématiquement, l'information mutuelle entre deux variables aléatoires discrètes X et Y est définie comme :
IM(X ; Y) = ∑∑ P(x, y) log( P(x, y) / (P(x) P(y)) )
où :
- P(x, y) is the distribution de probabilité conjointe de X et Y.
- P(x) is the probabilité marginale distribution de X.
- P(y) est la distribution de probabilité marginale de Y.
L'Information Mutuelle capture la réduction dans uncertainty about one variable given knowledge of the other. If X and Y are independent, MI(X; Y) equals zero, indicating no shared information. Conversely, a higher MI value indicates a stronger relationship and greater amount of shared information between the two variables.
In practical applications, MI is widely used in feature selection, where it helps identify the most informative features that contribute to a predictive model. It is also employed in clustering, l'enregistrement d'image, and analyzing the dependencies between random variables in complex systems.