Skip to content

Comparison 

Comparison(reference: str, comparison: str)

A comparison between two corpora: a reference corpus and a comparison corpus.

Parameters:

Name Type Description Default
reference str

Name of the reference corpus

 required
comparison str

Name of the comparison corpus

 required

Source

Methods:

Name Description
alias

Returns a new Comparison with the given alias, an internal name for it.
normalize

Returns a new Comparison whose token contributions are normalized onto a common scale.

Attributes:

Name Type Description
exclude ComparisonExcluder

Namespace for excluding tokens from the reference and comparison corpora.
score Scorer

Namespace for scoring comparisons between the reference and comparison corpora.

Attributes

exclude cached property 

exclude: ComparisonExcluder

Namespace for excluding tokens from the reference and comparison corpora.

See the available exclusion strategies.

Source

score cached property 

score: Scorer

Namespace for scoring comparisons between the reference and comparison corpora.

See the available scoring methods.

Source

Methods:

alias 

alias(name: str) -> Comparison

Returns a new Comparison with the given alias, an internal name for it.

Example 
import wordlevel as wl

wl.comp("before", "after").alias("my_comparison")

Parameters:

Name Type Description Default
name str

The name to give to the comparison

 required

Returns:

Type Description
Comparison

A new Comparison with the given alias

Source

normalize 

normalize(by: NormalizeBy) -> Comparison

Returns a new Comparison whose token contributions are normalized onto a common scale.

Each scoring measure produces a per-token contribution \(\delta_\tau\). Normalization rescales every contribution by a single constant so that scores become interpretable and comparable across measures. There are two normalization strategies:

  • "sum_abs" divides by the sum of absolute contributions, \(\sum_\tau |\delta_\tau|\). The absolute values of the normalized contributions then sum to 1.
  • "abs_sum" divides by the absolute sum of contributions, \(|\sum_\tau \delta_\tau|\). The normalized contributions then sum to \(\pm 1\).

Normalizing again by the same strategy is a no-op; normalizing by a different strategy raises an error, since the contributions are already on a normalized scale.

Example 
import wordlevel as wl

cl = wl.Catalog.from_dict_of_freqs(
    {
        "before": {"good": 12, "bad": 8, "happy": 6, "sad": 4},
        "after": {"good": 5, "bad": 7, "happy": 3, "sad": 9},
    }
)

cl = cl.with_comparisons(wl.comp("before", "after").score.shannon_entropy().normalize("sum_abs"))
Warning

Using this method forces the comparison to be evaluated eagerly rather than lazily, because validating it requires computing the scores over the full vocabulary.

Parameters:

Name Type Description Default
by NormalizeBy

The normalization strategy. "sum_abs" divides every token's contribution by \(\sum_\tau |\delta_\tau|\); "abs_sum" divides by \(|\sum_\tau \delta_\tau|\)

 required

Returns:

Type Description
Comparison

A new Comparison with normalized token contributions

Raises:

Type Description
ValueError

If the comparison is already normalized by a different strategy.
ValueError

If by is not a recognized normalization strategy.
ValueError

If by is "abs_sum" for a proportion comparison, whose contributions sum to zero by construction, so the absolute sum is always zero.

Source