相互情報量

相互情報量（MI） is a statistical measure that quantifies the amount of information obtained about one random variable through another random variable. It is particularly useful in fields like 情報理論, statistics, and 機械学習.

数学的には、二つの離散確率変数XとYの間の相互情報量は次のように定義されます：

MI(X; Y) = ∑∑ P(x, y) log( P(x, y) / (P(x) P(y)) )

ただし：

• P(x, y) is the 結合確率分布から XとYの。
• P(x) is the 周辺確率 Xの分布。
• P(y) はYの周辺確率分布です。

相互情報は、情報の削減を捉えます uncertainty about one variable given knowledge of the other. If X and Y are independent, MI(X; Y) equals zero, indicating no shared information. Conversely, a higher MI value indicates a stronger relationship and greater amount of shared information between the two variables.

In practical applications, MI is widely used in feature selection, where it helps identify the most informative features that contribute to a predictive model. It is also employed in clustering, 画像の位置合わせ, and analyzing the dependencies between random variables in complex systems.