onnx_diagnostic.torch_models.validate¶
- onnx_diagnostic.torch_models.validate.call_exporter(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False, dump_folder: str | None = None) Tuple[Dict[str, str | int | float], Dict[str, Any]] [source][source]¶
Calls an exporter on a model; If a patch must be applied, it should be before this functions.
- Parameters:
data – dictionary with all the necessary inputs
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
do_run – runs and compute discrepancies
dump_folder – to dump additional information
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.call_torch_export_custom(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, dump_folder: str | None = None) Tuple[Dict[str, Any], Dict[str, Any]] [source][source]¶
Exports a model into onnx. If a patch must be applied, it should be before this functions.
- Parameters:
data – dictionary with all the necessary inputs, the dictionary must contains keys
model
andinputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
dump_folder – to store additional information
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.call_torch_export_export(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False)[source][source]¶
Exports a model with
torch.export.export()
. If a patch must be applied, it should be before this functions.- Parameters:
data – dictionary with all the necessary inputs, the dictionary must contains keys
model
andinputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
do_run – runs and compute discrepancies
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.call_torch_export_model_builder(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None) Tuple[Dict[str, Any], Dict[str, Any]] [source][source]¶
Exports a model into onnx with ModelBuilder.
- Parameters:
data – dictionary with all the necessary inputs, the dictionary must contains keys
model
andinputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.call_torch_export_onnx(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None) Tuple[Dict[str, Any], Dict[str, Any]] [source][source]¶
Exports a model into onnx. If a patch must be applied, it should be before this functions.
- Parameters:
data – dictionary with all the necessary inputs, the dictionary must contains keys
model
andinputs_export
exporter – exporter to call
quiet – catch exception or not
verbose – verbosity
optimization – optimization to do
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.empty(value: Any) bool [source][source]¶
Tells if the value is empty.
- onnx_diagnostic.torch_models.validate.filter_inputs(inputs: Any, drop_names: List[str], model: Module | List[str] | None = None, dynamic_shapes: Any | None = None)[source][source]¶
Drops some inputs from the given inputs. It updates the dynamic shapes as well.
- onnx_diagnostic.torch_models.validate.get_inputs_for_task(task: str, config: Any | None = None) Dict[str, Any] [source][source]¶
Returns dummy inputs for a specific task.
- Parameters:
task – requested task
config – returns dummy inputs for a specific config if available
- Returns:
dummy inputs and dynamic shapes
- onnx_diagnostic.torch_models.validate.make_inputs(args: Tuple[Any, ...] | None, kwargs: Dict[str, Any] | None = None) Any [source][source]¶
Returns either args, kwargs or both depending on which ones are empty.
- onnx_diagnostic.torch_models.validate.run_ort_fusion(model_or_path: str | ModelProto, output_path: str, num_attention_heads: int, hidden_size: int, model_type: str = 'bert', verbose: int = 0) Tuple[Dict[str, Any], Dict[str, Any]] [source][source]¶
Runs onnxruntime fusion optimizer.
- Parameters:
model_or_path – path to the ModelProto or the ModelProto itself
output_path – the model to save
num_attention_heads – number of heads, usually
config.num_attention_heads
hidden_size – hidden size, usually
config.hidden_size
model_type – type of optimization, see below
verbose – verbosity
- Returns:
two dictionaries, summary and data
Supported values for
model_type
:<<<
import pprint from onnxruntime.transformers.optimizer import MODEL_TYPES pprint.pprint(sorted(MODEL_TYPES))
>>>
['bart', 'bert', 'bert_keras', 'bert_tf', 'clip', 'conformer', 'gpt2', 'gpt2_tf', 'gpt_neox', 'mmdit', 'phi', 'sam2', 'swin', 't5', 'tnlr', 'unet', 'vae', 'vit']
- onnx_diagnostic.torch_models.validate.shrink_config(cfg: Dict[str, Any]) Dict[str, Any] [source][source]¶
Shrinks the configuration before it gets added to the information to log.
- onnx_diagnostic.torch_models.validate.split_args_kwargs(inputs: Any) Tuple[Tuple[Any, ...], Dict[str, Any]] [source][source]¶
Splits into args, kwargs.
- onnx_diagnostic.torch_models.validate.validate_model(model_id: str, task: str | None = None, do_run: bool = False, exporter: str | None = None, do_same: bool = False, verbose: int = 0, dtype: str | dtype | None = None, device: str | device | None = None, same_as_pretrained: bool = False, use_pretrained: bool = False, optimization: str | None = None, quiet: bool = False, patch: bool | str | Dict[str, bool] = False, rewrite: bool = False, stop_if_static: int = 1, dump_folder: str | None = None, drop_inputs: List[str] | None = None, ortfusiontype: str | None = None, input_options: Dict[str, Any] | None = None, model_options: Dict[str, Any] | None = None, subfolder: str | None = None, opset: int | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1) Tuple[Dict[str, str | int | float], Dict[str, Any]] [source][source]¶
Validates a model. The function can also be called through the command line -m onnx_diagnostic validate … validate a model id.
- Parameters:
model_id – model id to validate
task – task used to generate the necessary inputs, can be left empty to use the default task for this model if it can be determined
do_run – checks the model works with the defined inputs
exporter – exporter the model using this exporter, available list:
export-strict
,export-nostrict
, … see belowdo_same – checks the discrepancies of the exported model
verbose – verbosity level
dtype – uses this dtype to check the model
device – do the verification on this device
same_as_pretrained – use a model equivalent to the trained, this is not always possible
use_pretrained – use the trained model, not the untrained one
optimization – optimization to apply to the exported model, depend on the the exporter
quiet – if quiet, catches exception if any issue
patch – applies patches (
patch_transformers=True, path_diffusers=True
) if True before exporting seeonnx_diagnostic.torch_export_patches.torch_export_patches()
, a string can be used to specify only one of themrewrite – applies known rewriting (
patch_transformers=True
) before exporting, seeonnx_diagnostic.torch_export_patches.torch_export_patches()
stop_if_static – stops if a dynamic dimension becomes static, see
onnx_diagnostic.torch_export_patches.torch_export_patches()
dump_folder – dumps everything in a subfolder of this one
drop_inputs – drops this list of inputs (given their names)
ortfusiontype – runs ort fusion, the parameters defines the fusion type, it accepts multiple values separated by
|
, seeonnx_diagnostic.torch_models.validate.run_ort_fusion()
input_options – additional options to define the dummy inputs used to export
model_options – additional options when creating the model such as
num_hidden_layers
orattn_implementation
subfolder – version or subfolders to uses when retrieving a model id
opset – onnx opset to use for the conversion
runtime – onnx runtime to use to check about discrepancies, only if do_run is true
repeat – number of time to measure the model
warmup – warmup the model first
inputs2 – checks that the second set of inputs is reunning as well, this ensures that the model does support dynamism, the value is used as an increment to the first set of values (added to dimensions)
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
The following environment variables can be used to print out some information:
PRINT_CONFIG
: prints the model configuration
The following exporters are available:
export-nostrict
: runtorch.export.export()
(…, strict=False)onnx-dynamo
: runtorch.onnx.export()
(…, dynamo=True), models can be optimized withoptimization
in("ir", "os_ort")
modelbuilder
: use ModelBuilder to builds the onnx modelcustom
: custom exporter (see experimental-experiment), models can be optimized withoptimization
in("default", "default+onnxruntime", "default+os_ort", "default+onnxruntime+os_ort")
The default runtime, onnxruntime is used to validate a model and check the exported model returns the same outputs as the original one, otherwise,
onnx_diagnostic.reference.TorchOnnxEvaluator
is used.
- onnx_diagnostic.torch_models.validate.validate_onnx_model(data: Dict[str, Any], quiet: bool = False, verbose: int = 0, flavour: str | None = None, runtime: str = 'onnxruntime', repeat: int = 1, warmup: int = 0, inputs2: int = 1) Tuple[Dict[str, Any], Dict[str, Any]] [source][source]¶
Verifies that an onnx model produces the same expected outputs. It uses
data["onnx_filename]
as the input onnx filename ordata["onnx_filename_{flavour}]
if flavour is specified.- Parameters:
data – dictionary with all the necessary inputs, the dictionary must contains keys
model
andinputs_export
quiet – catch exception or not
verbose – verbosity
flavour – use a different version of the inputs
runtime – onnx runtime to use, onnxruntime or torch
repeat – run that number of times the model
warmup – warmup the model
inputs2 – to validate the model on the second input set to make sure the exported model supports dynamism, the value is used as an increment added to the first set of inputs (added to dimensions)
- Returns:
two dictionaries, one with some metrics, another one with whatever the function produces
- onnx_diagnostic.torch_models.validate.version_summary() Dict[str, str | int | float] [source][source]¶
Example:
<<<
import pprint from onnx_diagnostic.torch_models.validate import version_summary pprint.pprint(version_summary())
>>>
{'version_date': '2025-07-04T18:26:01', 'version_numpy': '2.3.1', 'version_onnx': '1.19.0', 'version_onnx_diagnostic': '0.7.4', 'version_onnx_ir': '0.1.3', 'version_onnxruntime': '1.23.0', 'version_onnxscript': '0.3.0.dev20250301', 'version_scipy': '1.15.2', 'version_torch': '2.9.0.dev20250701+cu126', 'version_transformers': '4.54.0.dev0'}