lm_polygraph.estimators.token_sar module

class lm_polygraph.estimators.token_sar.TokenSAR[source]

Bases: Estimator

Estimates the sequence-level uncertainty of a language model following the method of “Token SAR” as provided in the paper https://arxiv.org/abs/2307.01379. Works only with whitebox models (initialized using lm_polygraph.utils.model.WhiteboxModel).

This method calculates the weighted sum of log_likelihoods with weights computed using token relevance.

lm_polygraph.estimators.token_sar.token_level_sar_scores(stats: Dict[str, ndarray]) → List[ndarray][source]