Skip to content

Comparison 

Comparison(reference: str, comparison: str)

A comparison between two corpora: a reference corpus and a comparison corpus.

Parameters:

Name Type Description Default
reference str

Name of the reference corpus

 required
comparison str

Name of the comparison corpus

 required

Source

Methods:

Name Description
alias

Returns a new Comparison with the given alias, an internal name for it.
normalize

Returns a new Comparison whose token contributions are normalized onto a common scale.

Attributes:

Name Type Description
exclude ComparisonExcluder

Namespace for excluding tokens from the reference and comparison corpora.
score Scorer

Namespace for scoring comparisons between the reference and comparison corpora.

Attributes

exclude cached property 

exclude: ComparisonExcluder

Namespace for excluding tokens from the reference and comparison corpora.

See the available exclusion strategies.

Source

score cached property 

score: Scorer

Namespace for scoring comparisons between the reference and comparison corpora.

See the available scoring methods.

Source

Methods:

alias 

alias(name: str) -> Comparison

Returns a new Comparison with the given alias, an internal name for it.

Example 
import wordlevel as wl

wl.comp("before", "after").alias("my_comparison")

Parameters:

Name Type Description Default
name str

The name to give to the comparison

 required

Returns:

Type Description
Comparison

A new Comparison with the given alias

Source

normalize 

normalize(
    by: NormalizeBy, check_nonzero: bool = True
) -> Comparison

Returns a new Comparison whose token contributions are normalized onto a common scale.

Example 
import wordlevel as wl

cl = wl.Catalog.from_dict_of_freqs(
    {
        "before": {"good": 12, "bad": 8, "happy": 6, "sad": 4},
        "after": {"good": 5, "bad": 7, "happy": 3, "sad": 9},
    }
)

cl = cl.with_comparisons(wl.comp("before", "after").score.shannon_entropy().normalize("total_diff"))

Each scoring measure produces a per-token contribution \(\delta_\tau\). Normalization divides every contribution by a single constant, either the total difference between the corpora \(\sum_\tau |\delta_\tau|\) or their net difference \(|\sum_\tau \delta_\tau|\), so that scores become interpretable and comparable across measures.

For what each strategy means and how to read a normalized score, see Normalization.

Warning

By default, using this method forces the comparison to be evaluated eagerly rather than lazily, because validating it requires computing the scores over the full vocabulary. The eager validation can be skipped using the check_nonzero parameter.

Parameters:

Name Type Description Default
by NormalizeBy

The normalization strategy. "total_diff" divides every token's contribution by the total difference \(\sum_\tau |\delta_\tau|\); "net_diff" divides by the net difference \(|\sum_\tau \delta_\tau|\)

 required
check_nonzero bool

Whether to check whether the normalization constant is zero and raise an error. If False, the operation remains lazy, but the scores can become NaN when normalizing by "total_diff" or inf and -inf when normalizing by "net_diff" if the normalization constant is zero

 True

Returns:

Type Description
Comparison

A new Comparison with normalized token contributions

Raises:

Type Description
ValueError

If check_nonzero is True and the normalization constant is zero. This is raised when the comparison is scored, not when normalize is called.
ValueError

If the comparison is already normalized by a different strategy.
ValueError

If by is not a recognized normalization strategy.
ValueError

If by is "net_diff" for a proportion comparison, whose contributions sum to zero by construction, so the net difference is always zero.

Source