Kullback–Leibler divergence

Rare card · No. 42791

In mathematical statistics, the Kullback–Leibler (KL) divergence (also called relative entropy and I-divergence), denoted D KL ( P ∥ Q ) {\displaystyle D_{\text{KL}}(P\parallel Q)} , is a type of statistical distance: a measure of how much an approximating probability distribution Q is different from a true probability distribution P. Mathematically, it is defined as D KL ( P ∥ Q ) = ∑ x ∈ X P ( x ) log ⁡ P ( x )...

Read on Wikipedia