Kullback-Leibler divergence

The Kullback–Leibler divergence is defined only if for all i, \(Q(i) = 0\) implies \(P(i) = 0\) (absolute continuity).

KL.divergence(p, q, base = 2)

Arguments

p	probability distribution P
q	probability distribution Q
base	baseof the logarithm

Value

numerical

Examples

data(Pi)
data <- getData(Pi)
data[1,3] <- 0.2
data[2,3] <- 0.1
Pi2 <- Distribution("Pi2")
setData(Pi2, data)
KL.divergence(Pi, Pi2)
#> [1] 0.1