lm_polygraph.generation_metrics.generation_metric module

class lm_polygraph.generation_metrics.generation_metric.GenerationMetric(**kwargs)

Bases: ABC

Abstract generation metric class, which measures ground-truth uncertainty by comparing model-generated text with dataset ground-truth text. This ground-truth uncertainty is further compared with different estimators’ uncertainties in UEManager using ue_metrics.