lm_polygraph.estimators.kernel_language_entropy module

class lm_polygraph.estimators.kernel_language_entropy.KernelLanguageEntropy(t: float = 0.3, normalize: bool = True, scale: bool = True, jitter: float = 0)

Bases: Estimator

Estimates the sequence-level uncertainty of a language model following the method of “Kernel Language Entropy” as provided in the paper https://arxiv.org/pdf/2405.20003 Works with both whitebox and blackbox models (initialized using lm_polygraph.utils.model.BlackboxModel/WhiteboxModel).

This method calculates KLE(Kheat) = VNE(Kheat), where VNE is von Neumann entropy and Kheat is a heat kernel of a semantic graph over language model’s outputs.

lm_polygraph.estimators.kernel_language_entropy.heat_kernel(laplacian: ndarray, t: float) → ndarray

lm_polygraph.estimators.kernel_language_entropy.laplacian_matrix(weighted_graph: ndarray) → ndarray

lm_polygraph.estimators.kernel_language_entropy.normalize_kernel(K: ndarray) → ndarray

lm_polygraph.estimators.kernel_language_entropy.scale_entropy(entropy: ndarray, n_classes: int) → ndarray

lm_polygraph.estimators.kernel_language_entropy.vn_entropy(K: ndarray, normalize: bool, scale: bool, jitter: float) → float64