lm_polygraph.estimators.conditional_pointwise_mutual_information module

class lm_polygraph.estimators.conditional_pointwise_mutual_information.ConditionalPointwiseMutualInformation(tau: float = 0.0656, lambd: float = 3.599)

Bases: Estimator

Estimates the token-level uncertainty of a language model following the method of Conditional Pointwise Mutual Information (CPMI) as provided in the paper https://arxiv.org/abs/2210.13210. Works only with whitebox models (initialized using lm_polygraph.utils.model.WhiteboxModel).

class lm_polygraph.estimators.conditional_pointwise_mutual_information.MeanConditionalPointwiseMutualInformation(tau: float = 0.0656, lambd: float = 3.599)

Bases: Estimator

Estimates the sequence-level uncertainty of a language model following the method of Conditional Pointwise Mutual Information (CPMI) as provided in the paper https://arxiv.org/abs/2210.13210. The sequence-level estimation is calculated as average token-level CPMI estimations. Works only with whitebox models (initialized using lm_polygraph.utils.model.WhiteboxModel).