lm_polygraph.stat_calculators.vllm_logprobs_extraction module

vLLM logprobs extraction calculator for lm-polygraph.

Extracts greedy_log_likelihoods and greedy_log_probs from vLLM output or from token_ids/logprobs directly (for truncated scoring). Unlike greedy_probs.py which both generates and extracts probabilities, this calculator only extracts probabilities from existing vLLM output.

class lm_polygraph.stat_calculators.vllm_logprobs_extraction.VLLMLogprobsExtractionCalculator(output_matrix: bool = False)

Bases: StatCalculator

Extracts greedy_log_likelihoods and greedy_log_probs from vLLM output or from token_ids/logprobs directly.

Args:

output_matrix: If True, output greedy_log_probs as 2D matrix [T, K]

for PDGap estimator. If False (default), output as list of 1D arrays for EntropyCalculator.

Usage:

# From vLLM output (original way) deps = {“vllm_output”: output} result = calculator(deps)

# From token_ids/logprobs directly (for truncated scoring) deps = {“token_ids”: truncated_ids, “logprobs”: truncated_logprobs} result = calculator(deps)

static meta_info()

Placeholder method to return the list of statistics and dependencies for the calculator.