Wechselseitige Information (MI) is a statistical measure that quantifies the amount of information obtained about one random variable through another random variable. It is particularly useful in fields like Informationstheorie, statistics, and maschinellem Lernen.
Mathematisch ist die wechselseitige Information zwischen zwei diskreten Zufallsvariablen X und Y definiert als:
MI(X; Y) = ∑∑ P(x, y) log( P(x, y) / (P(x) P(y)) )
wobei:
- P(x, y) is the Gemeinsame Wahrscheinlichkeitsverteilung von X und Y.
- P(x) is the Marginalwahrscheinlichkeit Verteilung von X.
- P(y) ist die marginale Wahrscheinlichkeitsverteilung von Y.
Gegenseitige Information erfasst die Reduktion in uncertainty about one variable given knowledge of the other. If X and Y are independent, MI(X; Y) equals zero, indicating no shared information. Conversely, a higher MI value indicates a stronger relationship and greater amount of shared information between the two variables.
In practical applications, MI is widely used in feature selection, where it helps identify the most informative features that contribute to a predictive model. It is also employed in clustering, Bildregistrierung, and analyzing the dependencies between random variables in complex systems.