onnx_diagnostic.torch_models.validate

onnx_diagnostic.torch_models.validate.call_exporter(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False, dump_folder: str | None = None) → Tuple[Dict[str, str | int | float], Dict[str, Any]][source][source]

Calls an exporter on a model; If a patch must be applied, it should be before this functions.

Parameters:

  • data – dictionary with all the necessary inputs

  • exporter – exporter to call

  • quiet – catch exception or not

  • verbose – verbosity

  • optimization – optimization to do

  • do_run – runs and compute discrepancies

  • dump_folder – to dump additional information

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_custom(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, dump_folder: str | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]

Exports a model into onnx. If a patch must be applied, it should be before this functions.

Parameters:

  • data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export

  • exporter – exporter to call

  • quiet – catch exception or not

  • verbose – verbosity

  • optimization – optimization to do

  • dump_folder – to store additional information

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_export(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False)[source][source]

Exports a model with torch.export.export(). If a patch must be applied, it should be before this functions.

Parameters:

  • data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export

  • exporter – exporter to call

  • quiet – catch exception or not

  • verbose – verbosity

  • optimization – optimization to do

  • do_run – runs and compute discrepancies

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_model_builder(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]

Exports a model into onnx with ModelBuilder.

Parameters:

  • data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export

  • exporter – exporter to call

  • quiet – catch exception or not

  • verbose – verbosity

  • optimization – optimization to do

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_onnx(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]

Exports a model into onnx. If a patch must be applied, it should be before this functions.

Parameters:

  • data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export

  • exporter – exporter to call

  • quiet – catch exception or not

  • verbose – verbosity

  • optimization – optimization to do

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.empty(value: Any) → bool[source][source]

Tells if the value is empty.

onnx_diagnostic.torch_models.validate.filter_inputs(inputs: Any, drop_names: List[str], model: Module | List[str] | None = None, dynamic_shapes: Any | None = None)[source][source]

Drops some inputs from the given inputs. It updates the dynamic shapes as well.

onnx_diagnostic.torch_models.validate.get_inputs_for_task(task: str, config: Any | None = None) → Dict[str, Any][source][source]

Returns dummy inputs for a specific task.

Parameters:

  • task – requested task

  • config – returns dummy inputs for a specific config if available

Returns:

dummy inputs and dynamic shapes

onnx_diagnostic.torch_models.validate.make_inputs(args: Tuple[Any, ...] | None, kwargs: Dict[str, Any] | None = None) → Any[source][source]

Returns either args, kwargs or both depending on which ones are empty.

onnx_diagnostic.torch_models.validate.run_ort_fusion(model_or_path: str | ModelProto, output_path: str, num_attention_heads: int, hidden_size: int, model_type: str = 'bert', verbose: int = 0) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]

Runs onnxruntime fusion optimizer.

Parameters:

  • model_or_path – path to the ModelProto or the ModelProto itself

  • output_path – the model to save

  • num_attention_heads – number of heads, usually config.num_attention_heads

  • hidden_size – hidden size, usually config.hidden_size

  • model_type – type of optimization, see below

  • verbose – verbosity

Returns:

two dictionaries, summary and data

Supported values for model_type:

<<<

import pprint
from onnxruntime.transformers.optimizer import MODEL_TYPES

pprint.pprint(sorted(MODEL_TYPES))

>>>

    ['bart',
     'bert',
     'bert_keras',
     'bert_tf',
     'clip',
     'conformer',
     'gpt2',
     'gpt2_tf',
     'gpt_neox',
     'mmdit',
     'phi',
     'sam2',
     'swin',
     't5',
     'tnlr',
     'unet',
     'vae',
     'vit']
onnx_diagnostic.torch_models.validate.shrink_config(cfg: Dict[str, Any]) → Dict[str, Any][source][source]

Shrinks the configuration before it gets added to the information to log.

onnx_diagnostic.torch_models.validate.split_args_kwargs(inputs: Any) → Tuple[Tuple[Any, ...], Dict[str, Any]][source][source]

Splits into args, kwargs.

onnx_diagnostic.torch_models.validate.validate_model(model_id: str, task: str | None = None, do_run: bool = False, exporter: str | None = None, do_same: bool = False, verbose: int = 0, dtype: str | dtype | None = None, device: str | device | None = None, same_as_pretrained: bool = False, use_pretrained: bool = False, optimization: str | None = None, quiet: bool = False, patch: bool | str | Dict[str, bool] = False, rewrite: bool = False, stop_if_static: int = 1, dump_folder: str | None = None, drop_inputs: List[str] | None = None, ortfusiontype: str | None = None, input_options: Dict[str, Any] | None = None, model_options: Dict[str, Any] | None = None, subfolder: str | None = None, opset: int | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1) → Tuple[Dict[str, str | int | float], Dict[str, Any]][source][source]

Validates a model. The function can also be called through the command line -m onnx_diagnostic validate … validate a model id.

Parameters:

  • model_id – model id to validate

  • task – task used to generate the necessary inputs, can be left empty to use the default task for this model if it can be determined

  • do_run – checks the model works with the defined inputs

  • exporter – exporter the model using this exporter, available list: export-strict, export-nostrict, … see below

  • do_same – checks the discrepancies of the exported model

  • verbose – verbosity level

  • dtype – uses this dtype to check the model

  • device – do the verification on this device

  • same_as_pretrained – use a model equivalent to the trained, this is not always possible

  • use_pretrained – use the trained model, not the untrained one

  • optimization – optimization to apply to the exported model, depend on the the exporter

  • quiet – if quiet, catches exception if any issue

  • patch – applies patches (patch_transformers=True, path_diffusers=True) if True before exporting see onnx_diagnostic.torch_export_patches.torch_export_patches(), a string can be used to specify only one of them

  • rewrite – applies known rewriting (patch_transformers=True) before exporting, see onnx_diagnostic.torch_export_patches.torch_export_patches()

  • stop_if_static – stops if a dynamic dimension becomes static, see onnx_diagnostic.torch_export_patches.torch_export_patches()

  • dump_folder – dumps everything in a subfolder of this one

  • drop_inputs – drops this list of inputs (given their names)

  • ortfusiontype – runs ort fusion, the parameters defines the fusion type, it accepts multiple values separated by |, see onnx_diagnostic.torch_models.validate.run_ort_fusion()

  • input_options – additional options to define the dummy inputs used to export

  • model_options – additional options when creating the model such as num_hidden_layers or attn_implementation

  • subfolder – version or subfolders to uses when retrieving a model id

  • opset – onnx opset to use for the conversion

  • runtime – onnx runtime to use to check about discrepancies, only if do_run is true

  • repeat – number of time to measure the model

  • warmup – warmup the model first

  • inputs2 – checks that the second set of inputs is reunning as well, this ensures that the model does support dynamism, the value is used as an increment to the first set of values (added to dimensions)

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

The following environment variables can be used to print out some information:

  • PRINT_CONFIG: prints the model configuration

The following exporters are available:

  • export-nostrict: run torch.export.export() (…, strict=False)

  • onnx-dynamo: run torch.onnx.export() (…, dynamo=True), models can be optimized with optimization in ("ir", "os_ort")

  • modelbuilder: use ModelBuilder to builds the onnx model

  • custom: custom exporter (see experimental-experiment), models can be optimized with optimization in ("default", "default+onnxruntime", "default+os_ort", "default+onnxruntime+os_ort")

The default runtime, onnxruntime is used to validate a model and check the exported model returns the same outputs as the original one, otherwise, onnx_diagnostic.reference.TorchOnnxEvaluator is used.

onnx_diagnostic.torch_models.validate.validate_onnx_model(data: Dict[str, Any], quiet: bool = False, verbose: int = 0, flavour: str | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]

Verifies that an onnx model produces the same expected outputs. It uses data["onnx_filename] as the input onnx filename or data["onnx_filename_{flavour}] if flavour is specified.

Parameters:

  • data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export

  • quiet – catch exception or not

  • verbose – verbosity

  • flavour – use a different version of the inputs

  • runtime – onnx runtime to use, onnxruntime or torch

  • repeat – run that number of times the model

  • warmup – warmup the model

  • inputs2 – to validate the model on the second input set to make sure the exported model supports dynamism, the value is used as an increment added to the first set of inputs (added to dimensions)

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.version_summary() → Dict[str, str | int | float][source][source]

Example:

<<<

import pprint
from onnx_diagnostic.torch_models.validate import version_summary

pprint.pprint(version_summary())

>>>

    {'version_date': '2025-07-04T18:26:01',
     'version_numpy': '2.3.1',
     'version_onnx': '1.19.0',
     'version_onnx_diagnostic': '0.7.4',
     'version_onnx_ir': '0.1.3',
     'version_onnxruntime': '1.23.0',
     'version_onnxscript': '0.3.0.dev20250301',
     'version_scipy': '1.15.2',
     'version_torch': '2.9.0.dev20250701+cu126',
     'version_transformers': '4.54.0.dev0'}