Skip to content

Catalog.from_dataframe

Catalog.from_dataframe classmethod 

from_dataframe(
    df: IntoFrame,
    token_col: str,
    corpora: Collection[str],
    normalize: bool = True,
) -> Catalog

Creates a Catalog of corpora from any dataframe supported by Narwhals.

The dataframe is converted to Polars before the catalog is constructed.

Warning

The catalog's engine is backed by Polars. Any dataframe it returns is a Polars dataframe, regardless of the backend that the data came from.

Polars dataframes are used as given, so a LazyFrame stays lazy. Dataframes from other lazy backends, like DuckDB and PySpark, are materialized in memory.

Example 
import pandas as pd
import wordlevel as wl

df = pd.DataFrame(
    {
        "token": ["good", "bad", "happy", "sad"],
        "before": [12, 8, 6, 4],
        "after": [5, 7, 3, 9],
    }
)

cl = wl.Catalog.from_dataframe(df, token_col="token", corpora=["before", "after"])

Parameters:

Name Type Description Default

df

IntoFrame

A dataframe containing corpus information, from any backend supported by Narwhals. Rows are tokens and columns are their frequencies in different corpora

 required

token_col

str

The column that contains the tokens (words, subwords, n-grams, etc.)

 required

corpora

Collection[str]

The columns that include the token frequencies for each corpus

 required

normalize

bool

Whether to normalize the corpus frequencies for each corpus so that they are in the range \([0, 1]\) and sum to 1. The original corpus frequencies will be preserved

 True

Returns:

Type Description
Catalog

A Catalog of corpora representing the provided tokens and frequencies

Raises:

Type Description
TypeError

If df is not a dataframe that can be converted to Polars.
KeyError

If token_col is not a column in df.
KeyError

If any column in corpora is not a column in df.
ValueError

If fewer than two columns are provided in corpora.

Source