Skip to content

Comparison.score.kullback_leibler_divergence

Comparison.score.kullback_leibler_divergence 

kullback_leibler_divergence(base: float = 2) -> Comparison

Calculates the Kullback-Leibler divergence (KLD) between the reference and comparison corpora.

Example 
import wordlevel as wl

cl = wl.Catalog.from_dict_of_freqs(
    {
        "before": {"good": 12, "bad": 8, "happy": 6, "sad": 4},
        "after": {"good": 5, "bad": 7, "happy": 3, "sad": 9},
    }
)

cl = cl.with_comparisons(wl.comp("before", "after").score.kullback_leibler_divergence())
Warning

Using this measure forces the comparison to be evaluated eagerly rather than lazily, because validating it requires computing over the full vocabulary.

The KLD is an asymmetric measure of how much more information is needed to encode the comparison corpus \(C\) when using the reference corpus \(R\) as the "codebook":

\[ \Phi = \sum_\tau p_\tau^{(C)} \log_b \frac{1}{p_\tau^{(R)}} - \sum_\tau p_\tau^{(C)} \log_b \frac{1}{p_\tau^{(C)}} \]

where \(p_\tau^{(R)}\) and \(p_\tau^{(C)}\) are the normalized frequencies of a token \(\tau\) in \(R\) and \(C\) respectively, and \(b\) is the logarithm base.

Reference

For more details, see the following paper:

Gallagher, R. J., Frank, M. R., Mitchell, L., Schwartz, A. J., Reagan, A. J., Danforth, C. M., & Dodds, P. S. (2021). Generalized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts. EPJ Data Science, 10(1), 4.

See Also
Score Description
Comparison.score.jensen_shannon_divergence Scores corpora using a bounded, symmetric, entropy-based divergence

Parameters:

Name Type Description Default

base

float

Base of the logarithms used to calculate the KLD

 2

Returns:

Type Description
Comparison

A new Comparison scored by the Kullback-Leibler divergence

Raises:

Type Description
ValueError

If the comparison corpus contains any token absent from the reference corpus, which causes the KLD to be infinite. Use a symmetric measure like the Jensen-Shannon divergence, or drop those tokens with Comparison.exclude.unshared_tokens

Source