lm_polygraph.generation_metrics.model_score module

class lm_polygraph.generation_metrics.model_score.ModelScoreSeqMetric[source]

Bases: GenerationMetric

Calculates sequence-level ModelScore metric between model-generated texts and ground truth texts. For each ground-truth text r and model-generated text ‘h’, method measures sum log-probabilitiy of generation ‘h’ on prompt ‘Paraphrase “{r}”’ normalized by the h length.

class lm_polygraph.generation_metrics.model_score.ModelScoreTokenwiseMetric[source]

Bases: GenerationMetric

Calculates token-level ModelScore metric between model-generated texts and ground truth texts. For each ground-truth text r and model-generated text ‘h’, method measures log-probabilities of generation ‘h’ on prompt ‘Paraphrase “{r}”’.