lm_polygraph.model_adapters.whitebox_model_basic module

class lm_polygraph.model_adapters.whitebox_model_basic.WhiteboxModelBasic(model: AutoModelForCausalLM, tokenizer: AutoTokenizer, tokenizer_args: Dict, generation_parameters=None, model_type='')

Bases: Model

Basic whitebox model adapter for using in stat calculators and uncertainty estimators.

device()

Returns the device the model is currently loaded on.

Returns:

str: device string.

generate(*args, **kwargs)

Generates output using the underlying model.

Args:

*args: Positional arguments to pass to model.generate() **kwargs: Keyword arguments to pass to model.generate(). These will override any

matching parameters from self.generation_parameters.

Returns:

The output from model.generate() with the combined generation parameters.

generate_texts(input_texts: List[str], **args)

Abstract method. Generates a list of model answers using input texts batch.

Parameters:

input_texts (List[str]): input texts batch.

Return:

List[str]: corresponding model generations. Have the same length as input_texts.

tokenize(texts: List[str], **kwargs) → Dict

Tokenizes input texts using the model’s tokenizer.

Args:

texts: List of input text strings to tokenize **kwargs: Additional arguments to pass to tokenizer

Returns:

Dict containing the tokenized inputs