onnx_diagnostic.torch_models.validate¶

onnx_diagnostic.torch_models.validate.call_exporter(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False, dump_folder: str | None = None, output_names: List[str] | None = None) → Tuple[Dict[str, str | int | float], Dict[str, Any]][source][source]¶

Calls an exporter on a model; If a patch must be applied, it should be before this functions.

Parameters:

data – dictionary with all the necessary inputs
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
do_run – runs and compute discrepancies
dump_folder – to dump additional information
output_names – list of output names to use with the onnx exporter

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_custom(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, dump_folder: str | None = None, output_names: List[str] | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]¶

Exports a model into onnx. If a patch must be applied, it should be before this functions.

Parameters:

data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
dump_folder – to store additional information
output_names – list of output names to use

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_export(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False)[source][source]¶

Exports a model with torch.export.export(). If a patch must be applied, it should be before this functions.

Parameters:

data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
do_run – runs and compute discrepancies

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_model_builder(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, output_names: List[str] | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]¶

Exports a model into onnx with ModelBuilder.

Parameters:

data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
output_names – list of output names to use

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.call_torch_export_onnx(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, output_names: List[str] | None = None) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]¶

Exports a model into onnx. If a patch must be applied, it should be before this functions.

Parameters:

data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
output_names – output names to use

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.empty(value: Any) → bool[source][source]¶: Tells if the value is empty.

onnx_diagnostic.torch_models.validate.filter_inputs(inputs: Any, drop_names: List[str], model: Module | List[str] | None = None, dynamic_shapes: Any | None = None)[source][source]¶: Drops some inputs from the given inputs. It updates the dynamic shapes as well.

onnx_diagnostic.torch_models.validate.get_inputs_for_task(task: str, config: Any | None = None) → Dict[str, Any][source][source]¶

Returns dummy inputs for a specific task.

Parameters:

task – requested task
config – returns dummy inputs for a specific config if available

Returns:

dummy inputs and dynamic shapes

onnx_diagnostic.torch_models.validate.make_inputs(args: Tuple[Any, ...] | None, kwargs: Dict[str, Any] | None = None) → Any[source][source]¶: Returns either args, kwargs or both depending on which ones are empty.

onnx_diagnostic.torch_models.validate.run_ort_fusion(model_or_path: str | ModelProto, output_path: str, num_attention_heads: int, hidden_size: int, model_type: str = 'bert', verbose: int = 0) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]¶

Runs onnxruntime fusion optimizer.

Parameters:

model_or_path – path to the ModelProto or the ModelProto itself
output_path – the model to save
num_attention_heads – number of heads, usually config.num_attention_heads
hidden_size – hidden size, usually config.hidden_size
model_type – type of optimization, see below
verbose – verbosity

Returns:

two dictionaries, summary and data

Supported values for model_type:

<<<

import pprint
from onnxruntime.transformers.optimizer import MODEL_TYPES

pprint.pprint(sorted(MODEL_TYPES))

>>>

    ['bart',
     'bert',
     'bert_keras',
     'bert_tf',
     'clip',
     'conformer',
     'gpt2',
     'gpt2_tf',
     'gpt_neox',
     'mmdit',
     'phi',
     'sam2',
     'swin',
     't5',
     'tnlr',
     'unet',
     'vae',
     'vit']

onnx_diagnostic.torch_models.validate.shrink_config(cfg: Dict[str, Any]) → Dict[str, Any][source][source]¶: Shrinks the configuration before it gets added to the information to log.

onnx_diagnostic.torch_models.validate.split_args_kwargs(inputs: Any) → Tuple[Tuple[Any, ...], Dict[str, Any]][source][source]¶: Splits into args, kwargs.

onnx_diagnostic.torch_models.validate.validate_model(model_id: str, task: str | None = None, do_run: bool = False, exporter: str | None = None, do_same: bool = False, verbose: int = 0, dtype: str | dtype | None = None, device: str | device | None = None, same_as_pretrained: bool = False, use_pretrained: bool = False, optimization: str | None = None, quiet: bool = False, patch: bool | str | Dict[str, bool] = False, rewrite: bool = False, stop_if_static: int = 1, dump_folder: str | None = None, drop_inputs: List[str] | None = None, ortfusiontype: str | None = None, input_options: Dict[str, Any] | None = None, model_options: Dict[str, Any] | None = None, subfolder: str | None = None, opset: int | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1, output_names: List[str] | None = None) → Tuple[Dict[str, str | int | float], Dict[str, Any]][source][source]¶

Validates a model. The function can also be called through the command line -m onnx_diagnostic validate … validate a model id.

Parameters:

model_id – model id to validate
task – task used to generate the necessary inputs, can be left empty to use the default task for this model if it can be determined
do_run – checks the model works with the defined inputs
exporter – exporter the model using this exporter, available list: export-strict, export-nostrict, … see below
do_same – checks the discrepancies of the exported model
verbose – verbosity level
dtype – uses this dtype to check the model
device – do the verification on this device
same_as_pretrained – use a model equivalent to the trained, this is not always possible
use_pretrained – use the trained model, not the untrained one
optimization – optimization to apply to the exported model, depend on the the exporter
quiet – if quiet, catches exception if any issue
patch – applies patches (patch_transformers=True, path_diffusers=True) if True before exporting see onnx_diagnostic.torch_export_patches.torch_export_patches(), a string can be used to specify only one of them
rewrite – applies known rewriting (patch_transformers=True) before exporting, see onnx_diagnostic.torch_export_patches.torch_export_patches()
stop_if_static – stops if a dynamic dimension becomes static, see onnx_diagnostic.torch_export_patches.torch_export_patches()
dump_folder – dumps everything in a subfolder of this one
drop_inputs – drops this list of inputs (given their names)
ortfusiontype – runs ort fusion, the parameters defines the fusion type, it accepts multiple values separated by |, see onnx_diagnostic.torch_models.validate.run_ort_fusion()
input_options – additional options to define the dummy inputs used to export
model_options – additional options when creating the model such as num_hidden_layers or attn_implementation
subfolder – version or subfolders to uses when retrieving a model id
opset – onnx opset to use for the conversion
runtime – onnx runtime to use to check about discrepancies, only if do_run is true
repeat – number of time to measure the model
warmup – warmup the model first
inputs2 – checks that the second set of inputs is reunning as well, this ensures that the model does support dynamism, the value is used as an increment to the first set of values (added to dimensions)
output_names – output names the onnx exporter should use

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

The following environment variables can be used to print out some information:

PRINT_CONFIG: prints the model configuration

The following exporters are available:

export-nostrict: run torch.export.export() (…, strict=False)
onnx-dynamo: run torch.onnx.export() (…, dynamo=True), models can be optimized with optimization in ("ir", "os_ort")
modelbuilder: use ModelBuilder to builds the onnx model
custom: custom exporter (see experimental-experiment), models can be optimized with optimization in ("default", "default+onnxruntime", "default+os_ort", "default+onnxruntime+os_ort")

The default runtime, onnxruntime is used to validate a model and check the exported model returns the same outputs as the original one, otherwise, onnx_diagnostic.reference.TorchOnnxEvaluator is used.

onnx_diagnostic.torch_models.validate.validate_onnx_model(data: Dict[str, Any], quiet: bool = False, verbose: int = 0, flavour: str | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1) → Tuple[Dict[str, Any], Dict[str, Any]][source][source]¶

Verifies that an onnx model produces the same expected outputs. It uses data["onnx_filename] as the input onnx filename or data["onnx_filename_{flavour}] if flavour is specified.

Parameters:

data – dictionary with all the necessary inputs, the dictionary must contains keys model and inputs_export
quiet – catch exception or not
verbose – verbosity
flavour – use a different version of the inputs
runtime – onnx runtime to use, onnxruntime or torch
repeat – run that number of times the model
warmup – warmup the model
inputs2 – to validate the model on the second input set to make sure the exported model supports dynamism, the value is used as an increment added to the first set of inputs (added to dimensions)

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.validate.version_summary() → Dict[str, str | int | float][source][source]¶

Example:

<<<

import pprint
from onnx_diagnostic.torch_models.validate import version_summary

pprint.pprint(version_summary())

>>>

    {'version_date': '2025-07-25T19:00:10',
     'version_numpy': '2.3.1',
     'version_onnx': '1.19.0',
     'version_onnx_diagnostic': '0.7.6',
     'version_onnx_ir': '0.1.5',
     'version_onnxruntime': '1.23.0',
     'version_onnxscript': '0.3.0.dev20250301',
     'version_scipy': '1.15.2',
     'version_torch': '2.9.0.dev20250723+cu126',
     'version_transformers': '4.54.0.dev0'}