lm_polygraph.stat_calculators package

Submodules

• lm_polygraph.stat_calculators.attention_eliciting_prompt module
  • AttentionElicitingPromptCalculator
    • AttentionElicitingPromptCalculator.meta_info()
  • BaseAttentionPromptCalculator
• lm_polygraph.stat_calculators.attention_forward_pass module
  • AttentionForwardPassCalculator
    • AttentionForwardPassCalculator.meta_info()
• lm_polygraph.stat_calculators.attention_forward_pass_visual module
  • AttentionForwardPassCalculatorVisual
    • AttentionForwardPassCalculatorVisual.meta_info()
• lm_polygraph.stat_calculators.bart_score module
  • BartScoreCalculator
    • BartScoreCalculator.load()
    • BartScoreCalculator.meta_info()
    • BartScoreCalculator.multi_ref_score()
    • BartScoreCalculator.score()
    • BartScoreCalculator.test()
• lm_polygraph.stat_calculators.claim_level_prompts module
• lm_polygraph.stat_calculators.cross_encoder_similarity module
  • CrossEncoderSimilarityMatrixCalculator
    • CrossEncoderSimilarityMatrixCalculator.meta_info()
• lm_polygraph.stat_calculators.cross_encoder_visual_similarity module
  • CrossEncoderSimilarityMatrixVisualCalculator
    • CrossEncoderSimilarityMatrixVisualCalculator.meta_info()
• lm_polygraph.stat_calculators.embeddings module
  • EmbeddingsCalculator
    • EmbeddingsCalculator.meta_info()
  • aggregate()
  • get_embeddings_from_output()
• lm_polygraph.stat_calculators.ensemble_token_data module
  • EnsembleTokenLevelDataCalculator
    • EnsembleTokenLevelDataCalculator.meta_info()
• lm_polygraph.stat_calculators.entropy module
  • EntropyCalculator
    • EntropyCalculator.meta_info()
• lm_polygraph.stat_calculators.extract_claims module
  • Claim
    • Claim.aligned_token_ids
    • Claim.claim_text
    • Claim.sentence
  • ClaimsExtractor
    • ClaimsExtractor.meta_info()
    • ClaimsExtractor.split_to_sentences()
• lm_polygraph.stat_calculators.greedy_alternatives_nli module
  • GreedyAlternativesFactPrefNLICalculator
    • GreedyAlternativesFactPrefNLICalculator.meta_info()
  • GreedyAlternativesNLICalculator
    • GreedyAlternativesNLICalculator.meta_info()
  • eval_nli_model()
• lm_polygraph.stat_calculators.greedy_cross_encoder_similarity module
  • GreedyCrossEncoderSimilarityMatrixCalculator
    • GreedyCrossEncoderSimilarityMatrixCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_lm_probs module
  • GreedyLMProbsCalculator
    • GreedyLMProbsCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_lm_visual_probs module
  • GreedyLMProbsVisualCalculator
    • GreedyLMProbsVisualCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_probs module
  • GreedyProbsCalculator
    • GreedyProbsCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_probs_blackbox module
  • BlackboxGreedyTextsCalculator
    • BlackboxGreedyTextsCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_semantic_matrix module
  • BaseGreedySemanticMatrixCalculator
    • BaseGreedySemanticMatrixCalculator.calculate_semantic_matrix()
  • ConcatGreedySemanticMatrixCalculator
    • ConcatGreedySemanticMatrixCalculator.meta_info()
  • GreedySemanticMatrixCalculator
    • GreedySemanticMatrixCalculator.meta_info()
• lm_polygraph.stat_calculators.greedy_visual_probs module
  • GreedyProbsVisualCalculator
    • GreedyProbsVisualCalculator.meta_info()
• lm_polygraph.stat_calculators.infer_causal_lm_calculator module
  • InferCausalLMCalculator
    • InferCausalLMCalculator.meta_info()
• lm_polygraph.stat_calculators.initial_state module
  • InitialStateCalculator
    • InitialStateCalculator.meta_info()
• lm_polygraph.stat_calculators.model_score module
  • ModelScoreCalculator
    • ModelScoreCalculator.meta_info()
• lm_polygraph.stat_calculators.prompt module
  • BasePromptCalculator
  • ClaimPromptCalculator
    • ClaimPromptCalculator.meta_info()
  • PromptCalculator
    • PromptCalculator.meta_info()
  • SamplingPromptCalculator
    • SamplingPromptCalculator.meta_info()
• lm_polygraph.stat_calculators.prompt_visual module
  • BasePromptVisualCalculator
  • ClaimPromptVisualCalculator
    • ClaimPromptVisualCalculator.meta_info()
  • PromptVisualCalculator
    • PromptVisualCalculator.meta_info()
  • SamplingPromptVisualCalculator
    • SamplingPromptVisualCalculator.meta_info()
• lm_polygraph.stat_calculators.raw_input module
  • RawInputCalculator
    • RawInputCalculator.meta_info()
• lm_polygraph.stat_calculators.sample module
  • BlackboxSamplingGenerationCalculator
    • BlackboxSamplingGenerationCalculator.meta_info()
  • OutputWrapper
    • OutputWrapper.decoder_hidden_states
    • OutputWrapper.encoder_hidden_states
    • OutputWrapper.hidden_states
  • SamplingGenerationCalculator
    • SamplingGenerationCalculator.meta_info()
• lm_polygraph.stat_calculators.sample_visual module
  • OutputWrapper
    • OutputWrapper.decoder_hidden_states
    • OutputWrapper.encoder_hidden_states
    • OutputWrapper.hidden_states
  • SamplingGenerationVisualCalculator
    • SamplingGenerationVisualCalculator.meta_info()
• lm_polygraph.stat_calculators.semantic_classes module
  • SemanticClassesCalculator
    • SemanticClassesCalculator.get_classes()
    • SemanticClassesCalculator.meta_info()
• lm_polygraph.stat_calculators.semantic_classes_claim_to_samples module
  • SemanticClassesClaimToSamplesCalculator
    • SemanticClassesClaimToSamplesCalculator.meta_info()
• lm_polygraph.stat_calculators.semantic_matrix module
  • SemanticMatrixCalculator
    • SemanticMatrixCalculator.meta_info()
• lm_polygraph.stat_calculators.stat_calculator module
  • StatCalculator
    • StatCalculator.meta_info()
    • StatCalculator.stat_dependencies
    • StatCalculator.stats
• lm_polygraph.stat_calculators.statistic_extraction module
  • TrainingStatisticExtractionCalculator
    • TrainingStatisticExtractionCalculator.meta_info()
• lm_polygraph.stat_calculators.statistic_extraction_visual module
  • TrainingStatisticExtractionCalculatorVisual
    • TrainingStatisticExtractionCalculatorVisual.meta_info()
• lm_polygraph.stat_calculators.vllm_logprobs_extraction module
  • VLLMLogprobsExtractionCalculator
    • VLLMLogprobsExtractionCalculator.meta_info()