lm_polygraph.generation_metrics.openai_fact_check module

class lm_polygraph.generation_metrics.openai_fact_check.OpenAIFactCheck(llm_url: str = None, openai_model: str = 'gpt-4o', cache_path: str = '/home/docs/.cache', language: str = 'en', progress_bar: bool = False, fact_check_prompts: Dict[str, str] = {'ar': 'السؤال: {input}\n\nحدد ما إذا كانت جميع المعلومات المقدمة في الادعاء التالي صحيحة وفقًا لأحدث مصادر المعلومات.\n\nالادعاء: {claim}\n', 'de': 'Frage: {input}\n\nStellen Sie fest, ob alle bereitgestellten Informationen in der folgenden Behauptung gemäß den neuesten Informationsquellen wahr sind.\n\nBehauptung: {claim}\n', 'en': 'Question: {input}\n\nDetermine if all provided information in the following claim is true according to the most recent sources of information.\n\nClaim: {claim}\n', 'ru': 'Question: {input}\n\n    Определи, соответствует ли вся предоставленная информация в следующем утверждении действительности согласно самым последним источникам информации.\n    Если хотя бы часть утверждения неверна, склоняйся к выводу, что информация ложная.\n    Think in English.\n    Think step by step on how to summarize the claim within the provided <sketchpad>.\n    Then, return a <summary> based on the <sketchpad>.\n\n    Claim: {claim}\n', 'zh': 'Question: {input}\n\n请根据最新的信息来源，确定以下声明（Claim）中提供的所有信息是否属实。\n\nClaim: {claim}\n'}, fact_check_summarize_prompt: Dict[str, str] = {'ar': '\nالسؤال: {input}\n\nالادعاء: {claim}\n\nهل الادعاء التالي صحيح؟\n\nالاجابة: {reply}\n\nقم بتلخيص هذه الجملة في كلمة واحدة، سواء كان الادعاء صحيح: "صحيح" أو "خطأ" أو "غير معروف".\n', 'de': 'Question: {input}\n\nClaim: {claim}\n\nIs the following claim true?\n\nReply: {reply}\n\nSummarize this reply into one word, whether the claim is true: "True", "False" or "Not known".', 'en': 'Question: {input}\n\nClaim: {claim}\n\nIs the following claim true?\n\nReply: {reply}\n\nSummarize this reply into one word, whether the claim is true: "True", "False" or "Not known".\n', 'ru': 'Question: {input}\n\nClaim: {claim}\n\nIs the following claim true?\n\nReply: {reply}\n\nSummarize this reply into one word, whether the claim is true: "True", "False" or "Not known".\n', 'zh': 'Question: {input}\n\nClaim: {claim}\n\n以下的表述是否正确？\n\nReply: {reply}\n\n请用一个词回答该表述(Reply)是否正确："True"，"False"或"Not known"。\n'}, n_threads: int = 1, timeout: int = 600, max_tokens: int = None, rewrite_cache: bool = False)

Bases: GenerationMetric

Calculates for each claim, whether it is true of not, using OpenAI model specified in lm_polygraph.stat_calculators.openai_chat.OpenAIChat.