onnx_diagnostic.torch_models.hghub.model_inputs

onnx_diagnostic.torch_models.hghub.model_inputs.check_hasattr(config: Any, *args: str | Tuple[Any, ...])[source]

Checks the confiugation has all the attributes in args. Raises an exception otherwise.

onnx_diagnostic.torch_models.hghub.model_inputs.compute_model_size(model: Module) Tuple[int, int][source]

Returns the size of the models (weights only) and the number of the parameters.

onnx_diagnostic.torch_models.hghub.model_inputs.config_class_from_architecture(arch: str, exc: bool = False) type | None[source]

Retrieves the configuration class for a given architecture.

Parameters:
  • arch – architecture (clas name)

  • exc – raise an exception if not found

Returns:

type

onnx_diagnostic.torch_models.hghub.model_inputs.filter_out_unexpected_inputs(model: Module, kwargs: Dict[str, Any])[source]

Removes input names in kwargs if no parameter names was found in model.forward.

onnx_diagnostic.torch_models.hghub.model_inputs.get_inputs_for_image_classification(model: Module, config: Any | None, input_width: int, input_height: int, input_channels: int, batch_size: int = 2, dynamic_rope: bool = False, **kwargs)[source]

Generates inputs for task image-classification.

Parameters:
  • model – model to get the missing information

  • config – configuration used to generate the model

  • batch_size – batch size

  • input_channel – input channel

  • input_width – input width

  • input_height – input height

  • kwargs – to overwrite the configuration, example num_hidden_layers=1

Returns:

dictionary

onnx_diagnostic.torch_models.hghub.model_inputs.get_inputs_for_text2text_generation(model: Module, config: Any | None, dummy_max_token_id: int, num_key_value_heads: int, num_hidden_layers: int, head_dim: int, encoder_dim: int, batch_size: int = 2, sequence_length: int = 30, sequence_length2: int = 3, **kwargs)[source]

Generates input for task text2text-generation.

Parameters:
  • model – model to get the missing information

  • config – configuration used to generate the model

  • head_dim – last dimension of the cache

  • dummy_max_token_id – dummy max token id

  • batch_size – batch size

  • encoder_dim – last dimension of encoder_last_hidden_state

  • sequence_length – sequence length

  • sequence_length2 – new sequence length

  • kwargs – to overwrite the configuration, example num_hidden_layers=1

Returns:

dictionary

Stolen inputs for one model.

cache_position:T7s1
past_key_values:EncoderDecoderCache(
    self_attention_cache=DynamicCache(
        key_cache=#6[T1s1x8x1x64,...],
        value_cache=#6[T1s1x8x1x64,...]),
    cross_attention_cache=DynamicCache(
        key_cache=#6[T1s1x8x16x64,...],
        value_cache=#6[T1s1x8x16x64,...])),
decoder_input_ids:T7s1x1,
encoder_outputs:dict(last_hidden_state:T1s1x16x512)
onnx_diagnostic.torch_models.hghub.model_inputs.get_inputs_for_text_generation(model: Module, config: Any | None, dummy_max_token_id: int, num_key_value_heads: int, num_hidden_layers: int, head_dim: int, batch_size: int = 2, sequence_length: int = 30, sequence_length2: int = 3, dynamic_rope: bool = False, **kwargs)[source]

Generates input for task text-generation.

Parameters:
  • model – model to get the missing information

  • config – configuration used to generate the model

  • head_dim – last dimension of the cache

  • dummy_max_token_id – dummy max token id

  • batch_size – batch size

  • sequence_length – sequence length

  • sequence_length2 – new sequence length

  • dynamic_rope – use dynamic rope (see transformers.LlamaConfig)

  • kwargs – to overwrite the configuration, example num_hidden_layers=1

Returns:

dictionary

onnx_diagnostic.torch_models.hghub.model_inputs.random_input_kwargs(config: Any, task: str) Tuple[Dict[str, Any], Callable][source]

Inputs kwargs

onnx_diagnostic.torch_models.hghub.model_inputs.reduce_model_config(config: Any, task: str) Dict[str, Any][source]

Reduces a model size.