Comparison.score.jensen_shannon_divergence

Comparison.score.jensen_shannon_divergence ¶

jensen_shannon_divergence(
    alpha: float = 1,
    base: float = 2,
    weight_ref: float = 0.5,
    weight_cmp: float = 0.5,
    reference_score: float | Literal["mean"] | Expr = 0,
) -> Comparison

Calculates the Jensen-Shannon divergence (JSD) between the reference and comparison corpora.

Example

import wordlevel as wl

cl = wl.Catalog.from_dict_of_freqs(
    {
        "before": {"good": 12, "bad": 8, "happy": 6, "sad": 4},
        "after": {"good": 5, "bad": 7, "happy": 3, "sad": 9},
    }
)

cl = cl.with_comparisons(wl.comp("before", "after").score.jensen_shannon_divergence())

The JSD is a symmetric, bounded measure of how two corpora differ. It averages the Kullback-Leibler divergence of the reference corpus \(R\) and the comparison corpus \(C\) against a synthetic mixture corpus \(M\) representing the combination of both. When \(\alpha = 1\):

\[ \begin{aligned} \Phi &= \pi_R \, D_{\text{KL}} \left( R \parallel M \right) + \pi_C \, D_{\text{KL}} \left( C \parallel M \right) \\[6pt] &= \sum_\tau p_\tau^{(M)} \log_b \frac{1}{p_\tau^{(M)}} - \left( \pi_R \sum_\tau p_\tau^{(R)} \log_b \frac{1}{p_\tau^{(R)}} + \pi_C \sum_\tau p_\tau^{(C)} \log_b \frac{1}{p_\tau^{(C)}} \right) \end{aligned} \]

where \(p_\tau^{(M)} = \pi_R \, p_\tau^{(R)} + \pi_C \, p_\tau^{(C)}\) is the normalized frequency of a token \(\tau\) in the mixture, with weights \(\pi_R\) and \(\pi_C\) such that \(\pi_R + \pi_C = 1\), and \(b\) is the logarithm base.

Reference

For more details, see the following paper:

Gallagher, R. J., Frank, M. R., Mitchell, L., Schwartz, A. J., Reagan, A. J., Danforth, C. M., & Dodds, P. S. (2021). Generalized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts. EPJ Data Science, 10(1), 4.

See Also

Score	Description
`Comparison.score.kullback_leibler_divergence`	Scores corpora using an asymmetric entropy-based divergence

Parameters:

Name	Type	Description	Default
`alpha` ¶	`float`	Order of the generalized entropy. \(\alpha = 1\) recovers the standard JSD. Larger values place more weight on common tokens, and smaller values place more weight on rare tokens	`1`
`base` ¶	`float`	Base of the logarithms used to calculate the JSD when \(\alpha = 1\). Ignored otherwise	`2`
`weight_ref` ¶	`float`	Mixture weight \(\pi_R\) for the reference corpus	`0.5`
`weight_cmp` ¶	`float`	Mixture weight \(\pi_C\) for the comparison corpus	`0.5`
`reference_score` ¶	`float \| Literal['mean'] \| Expr`	Baseline against which per-token contributions are interpreted. Defaults to 0. If `"mean"`, uses the weighted average of the reference corpus's token scores as the reference score	`0`

Returns:

Type	Description
`Comparison`	A new `Comparison` scored by the Jensen-Shannon divergence

Source

Comparison.score.jensen_shannon_divergence