lm_polygraph.ue_metrics package

Submodules

lm_polygraph.ue_metrics.kendalltau module

class lm_polygraph.ue_metrics.kendalltau.KendallTauCorrelation[source]

Bases: UEMetric

Calculates the Kendall’s tau correlation.

lm_polygraph.ue_metrics.pr_auc module

class lm_polygraph.ue_metrics.pr_auc.PRAUC(positive_class: int = 1, negative_class: int = 0)[source]

Bases: UEMetric

preprocess_inf(x, array)[source]

lm_polygraph.ue_metrics.pred_rej_area module

class lm_polygraph.ue_metrics.pred_rej_area.PredictionRejectionArea(max_rejection: float = 1.0)[source]

Bases: UEMetric

Calculates area under Prediction-Rejection curve.

lm_polygraph.ue_metrics.rev_pairs_prop module

class lm_polygraph.ue_metrics.rev_pairs_prop.ReversedPairsProportion[source]

Bases: UEMetric

Calculates Reversed Pairs Proportion metrics. For uncetainty estimations e and ground-truth uncertainties g, the class calculates the proportion of pairs (i, j), such that e[i] < e[j] and g[i] > g[j].

lm_polygraph.ue_metrics.risk_cov_curve module

class lm_polygraph.ue_metrics.risk_cov_curve.RiskCoverageCurveAUC(normalize: bool = True)[source]

Bases: UEMetric

Calculates area under the Risk-Coverage curve.

lm_polygraph.ue_metrics.roc_auc module

class lm_polygraph.ue_metrics.roc_auc.ROCAUC[source]

Bases: UEMetric

is_ood_metric = True
preprocess_inf(x, array)[source]

lm_polygraph.ue_metrics.spearmanr module

class lm_polygraph.ue_metrics.spearmanr.SpearmanRankCorrelation[source]

Bases: UEMetric

Calculates the Spearman’s rank correlation coefficient.

lm_polygraph.ue_metrics.ue_metric module

class lm_polygraph.ue_metrics.ue_metric.UEMetric[source]

Bases: ABC

Abstract class, which measures the quality of uncertainty estimations from some Estimator using ground-truth uncertainty estimations calculated from some GenerationMetric.

lm_polygraph.ue_metrics.ue_metric.get_random_scores(function, metrics, num_iter=1000, seed=42)[source]
lm_polygraph.ue_metrics.ue_metric.normalize(target: List[float])[source]
lm_polygraph.ue_metrics.ue_metric.normalize_metric(target_score, oracle_score, random_score)[source]
lm_polygraph.ue_metrics.ue_metric.skip_target_nans(target, estimator)[source]

Module contents