lm_polygraph.estimators.eccentricity module

class lm_polygraph.estimators.eccentricity.Eccentricity(similarity_score: Literal['NLI_score', 'Jaccard_score'] = 'NLI_score', affinity: Literal['entail', 'contra'] = 'entail', verbose: bool = False, thres: float = 0.9)

Bases: Estimator

Estimates the sequence-level uncertainty of a language model following the method of “Eccentricity” as provided in the paper https://arxiv.org/abs/2305.19187. Works with both whitebox and blackbox models (initialized using lm_polygraph.utils.model.BlackboxModel/WhiteboxModel).

Method calculates a frobenious (euclidian) norm between all eigenvectors that are informative embeddings of graph Laplacian (lower norm -> closer embeddings -> higher eigenvectors -> greater uncertainty).

U_Eccentricity(i, stats)