lm_polygraph.estimators.verbalized_1s module

class lm_polygraph.estimators.verbalized_1s.Verbalized1S(confidence_regex='', name_postfix='')

Bases: Estimator

Estimates sequence-level uncertainty of a language model by extracting the confidence estimate from the model’s answer using a provided regex. To use this estimator, model has to be correctly prompted to output it’s confidence in the answer. Adapted from the original implementation in the paper https://arxiv.org/abs/2305.14975