lm_polygraph.utils.model module

class lm_polygraph.utils.model.BlackboxModel(openai_api_key: str = None, model_path: str = None, hf_api_token: str = None, generation_parameters: GenerationParameters = GenerationParameters(temperature=1.0, top_k=50, top_p=1.0, do_sample=False, num_beams=1, presence_penalty=0.0, repetition_penalty=1.0, stop_strings=None, allow_newlines=True, max_new_tokens=100), supports_logprobs: bool = False)

Bases: Model

Black-box model class. Have no access to model scores and logits. Currently implemented blackbox models: OpenAI models, Huggingface models.

Examples:

`python >>> from lm_polygraph import BlackboxModel >>> model = BlackboxModel.from_openai( ...     'YOUR_OPENAI_TOKEN', ...     'gpt-3.5-turbo' ... ) `

`python >>> from lm_polygraph import BlackboxModel >>> model = BlackboxModel.from_huggingface( ...     hf_api_token='YOUR_API_TOKEN', ...     hf_model_id='google/t5-large-ssm-nqo' ... ) `

static from_huggingface(hf_api_token: str, hf_model_id: str, **kwargs)

Initializes a blackbox model from huggingface.

Parameters:

hf_api_token (Optional[str]): Huggingface API token if the blackbox model comes from HF. Default: None. hf_model_id (Optional[str]): model path in huggingface.

static from_openai(openai_api_key: str, model_path: str, supports_logprobs: bool = False, **kwargs)

Initializes a blackbox model from OpenAI API.

Parameters:

openai_api_key (Optional[str]): OpenAI API key. Default: None. model_path (Optional[str]): model name in OpenAI. supports_logprobs (bool): Whether the model supports returning log probabilities. Default: False.

generate(**args)

For OpenAI models with logprobs support, returns a lightweight wrapper around OpenAI API response. For other blackbox models, raises an exception as this is not implemented.

Parameters:

**args: Arguments to pass to the generate method.

Returns:

object: A wrapper around the OpenAI API response if logprobs are supported.

Raises:

Exception: If the model doesn’t support logprobs.

generate_texts(input_texts: List[str], **args) → List[str]

Generates a list of model answers using input texts batch.

Parameters:

input_texts (List[str]): input texts batch.

Return:

List[str]: corresponding model generations. Have the same length as input_texts.

tokenizer(*args, **kwargs)

Not implemented for blackbox models.

class lm_polygraph.utils.model.Model(model_path: str, model_type: str)

Bases: ABC

Abstract model class. Used as base class for both White-box models and Black-box models.

abstract generate(**args)

Abstract method. Generates the model output with scores from batch formed by HF Tokenizer. Not implemented for black-box models.

abstract generate_texts(input_texts: List[str], **args) → List[str]

Abstract method. Generates a list of model answers using input texts batch.

Parameters:

input_texts (List[str]): input texts batch.

Return:

List[str]: corresponding model generations. Have the same length as input_texts.

class lm_polygraph.utils.model.WhiteboxModel(model: AutoModelForCausalLM, tokenizer: AutoTokenizer, model_path: str = None, model_type: str = 'CausalLM', generation_parameters: GenerationParameters = GenerationParameters(temperature=1.0, top_k=50, top_p=1.0, do_sample=False, num_beams=1, presence_penalty=0.0, repetition_penalty=1.0, stop_strings=None, allow_newlines=True, max_new_tokens=100), instruct: bool = False)

Bases: Model

White-box model class. Have access to model scores and logits. Currently implemented only for Huggingface models.

Examples:

`python >>> from lm_polygraph import WhiteboxModel >>> model = WhiteboxModel.from_pretrained( ...     "bigscience/bloomz-3b", ... ) `

device()

Returns the device the model is currently loaded on.

Returns:

str: device string.

static from_pretrained(model_path: str, generation_params: Dict | None = {}, add_bos_token: bool = True, **kwargs)

Initializes the model from HuggingFace. Automatically determines model type.

Parameters:

model_path (str): model path in HuggingFace. generation_params (Dict): generation arguments for

lm_polygraph.utils.generation_parametersGenerationParameters

add_bos_token (bool): tokenizer argument. Default: True.

generate(**args)

Generates the model output with scores from batch formed by HF Tokenizer.

Parameters:

**args: Any arguments that can be passed to model.generate function from HuggingFace.

Returns:

ModelOutput: HuggingFace generation output with scores overriden with original probabilities.

generate_texts(input_texts: List[str], **args) → List[str]

Generates a list of model answers using input texts batch.

Parameters:

input_texts (List[str]): input texts batch.

Return:

List[str]: corresponding model generations. Have the same length as input_texts.

tokenize(texts: List[str] | List[List[Dict[str, str]]]) → Dict[str, Tensor]

Tokenizes input texts batch into a dictionary using the model tokenizer.

Parameters:

texts (List[str]): list of input texts batch.

Returns:

dict[str, torch.Tensor]: tensors dictionary obtained by tokenizing input texts batch.

lm_polygraph.utils.model.create_ensemble(models: List[WhiteboxModel] = [], mc: bool = False, seed: int = 1, mc_seeds: List[int] = [1], ensembling_mode: str = 'pe', dropout_rate: float = 0.1, **kwargs) → WhiteboxModel