lm_polygraph.ue_metrics.ue_metric module

class lm_polygraph.ue_metrics.ue_metric.UEMetric[source]

Bases: ABC

Abstract class, which measures the quality of uncertainty estimations from some Estimator using ground-truth uncertainty estimations calculated from some GenerationMetric.

lm_polygraph.ue_metrics.ue_metric.get_random_scores(function, metrics, num_iter=1000, seed=42)[source]

lm_polygraph.ue_metrics.ue_metric.normalize(target: List[float])[source]

lm_polygraph.ue_metrics.ue_metric.normalize_metric(target_score, oracle_score, random_score)[source]

lm_polygraph.ue_metrics.ue_metric.skip_target_nans(target, estimator)[source]