Comparison.score.jensen_shannon_divergence

Comparison.score.jensen_shannon_divergence ¶

jensen_shannon_divergence(
    alpha: float = 1,
    base: float = 2,
    weight_ref: float = 0.5,
    weight_cmp: float = 0.5,
    reference_score: float | Literal["mean"] | Expr = 0,
) -> Comparison

Calculates the Jensen-Shannon divergence (JSD) between the reference and comparison corpora.

The JSD is a symmetric, bounded measure of how two corpora differ. It is computed by averaging the Kullback-Leibler divergence of each corpus against a synthetic mixture corpus representing the combination of both corpora. This implementation generalizes the JSD through an order parameter \(\alpha\). When \(\alpha < 1\), more weight is given to rare tokens; when \(\alpha > 1\), more weight is given to common tokens.

Example

import wordlevel as wl

cl = wl.Catalog.from_dict_of_freqs(
    {
        "before": {"good": 12, "bad": 8, "happy": 6, "sad": 4},
        "after": {"good": 5, "bad": 7, "happy": 3, "sad": 9},
    }
)

cl = cl.with_comparisons(wl.comp("before", "after").score.jensen_shannon_divergence())

When the reference_score is 0 and \(\alpha = 1\), the contribution \(\delta_\tau\) of a token \(\tau\) is:

\[ \delta_\tau = p_\tau^{(M)} \log \frac{1}{p_\tau^{(M)}} - \left( \pi_R \, p_\tau^{(R)} \log \frac{1}{p_\tau^{(R)}} + \pi_C \, p_\tau^{(C)} \log \frac{1}{p_\tau^{(C)}} \right) \]

where \(p_\tau^{(R)}\) and \(p_\tau^{(C)}\) are the normalized frequencies of token \(\tau\) in \(R\) and \(C\) respectively, the reference and comparison corpora, and \(p_\tau^{(M)} = \pi_R \, p_\tau^{(R)} + \pi_C \, p_\tau^{(C)}\) with weights \(\pi_R\) and \(\pi_C\) such that \(\pi_R + \pi_C = 1\). For \(\alpha = 1\), each token's contribution is always non-negative.

When the reference_score is 0 and \(\alpha \neq 1\), the contribution of the token is:

\[ \delta_\tau = \left( p_\tau^{(C)} - p_\tau^{(R)} \right) \left[ \frac{1}{2} \left( \phi_\tau^{(C)} + \phi_\tau^{(R)} \right) - \Phi^{(\text{ref})} \right] + \frac{1}{2} \left( p_\tau^{(C)} + p_\tau^{(R)} \right) \left( \phi_\tau^{(C)} - \phi_\tau^{(R)} \right) \]

where \(\Phi^{(\text{ref})}\) is the user-provided reference score, and

\[ \phi_\tau^{(C)} = \pi_C \left( \frac {\left[ p_\tau^{(C)} \right]^{\alpha - 1} - \left[ p_\tau^{(M)} \right]^{\alpha - 1}} {\alpha - 1} \right) \, , \, \phi_\tau^{(R)} = \pi_R \left( \frac {\left[ p_\tau^{(M)} \right]^{\alpha - 1} - \left[ p_\tau^{(R)} \right]^{\alpha - 1}} {\alpha - 1} \right) \]

Reference

For more details, see the following paper:

Gallagher, R. J., Frank, M. R., Mitchell, L., Schwartz, A. J., Reagan, A. J., Danforth, C. M., & Dodds, P. S. (2021). Generalized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts. EPJ Data Science, 10(1), 4.

See Also

Score	Description
`Comparison.score.kullback_leibler_divergence`	Scores corpora using an asymmetric entropy-based divergence

Parameters:

Name	Type	Description	Default
`alpha` ¶	`float`	Order of the generalized entropy. \(\alpha = 1\) recovers the standard JSD. Larger values place more weight on common tokens, and smaller values place more weight on rare tokens	`1`
`base` ¶	`float`	Base of the logarithms used to calculate the JSD when \(\alpha = 1\). Ignored otherwise	`2`
`weight_ref` ¶	`float`	Mixture weight \(\pi_R\) for the reference corpus	`0.5`
`weight_cmp` ¶	`float`	Mixture weight \(\pi_C\) for the comparison corpus	`0.5`
`reference_score` ¶	`float \| Literal['mean'] \| Expr`	Baseline against which per-token contributions are interpreted. Defaults to 0. If `"mean"`, uses the weighted average of the reference corpus, where weights are the values described in the general formula above	`0`

Returns:

Type	Description
`Comparison`	A new `Comparison` scored by the Jensen-Shannon divergence

Source

Comparison.score.jensen_shannon_divergence