Information Theory

Algorithmic probability quantifies the likelihood of a string appearing based on its shortest description.

Entropy is a measure of uncertainty or disorder in a system, often used in thermodynamics and information theory.

Flux refers to the flow or transfer of energy, matter, or information in physics and other fields.

Hamming Distance measures the difference between two strings of equal length.

Information Gain measures the reduction in uncertainty about a random variable given additional information.

Information Theory studies the quantification, storage, and communication of information.

Jensen-Shannon Divergence measures the similarity between two probability distributions.

Joint entropy measures the uncertainty of two random variables together.

JSDivergence measures the similarity between two probability distributions using a symmetric approach.

K-L Divergence measures how one probability distribution differs from a second, reference distribution.

Maximum Entropy is a statistical principle used to make predictions based on limited information.

Mutual Information measures the amount of information shared between two variables.

A method for estimating mutual information using neural networks, enhancing data dependence measurement.