onnx_diagnostic.torch_models.test_helper

onnx_diagnostic.torch_models.test_helper.call_exporter(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False) → Tuple[Dict[str, int | float | str], Dict[str, Any]]

Calls an exporter on a model; If a patch must be applied, it should be before this functions.

Parameters:

• data – dictionary with all the necessary inputs

• exporter – exporter to call

• quiet – catch exception or not

• verbose – verbosity

• patch – apply patches

• optimization – optimization to do

• do_run – runs and compute discrepancies

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.test_helper.call_torch_export_export(data: Dict[str, Any], exporter: str, quiet: bool = False, verbose: int = 0, optimization: str | None = None, do_run: bool = False)

Calls an exporter on a model; If a patch must be applied, it should be before this functions.

Parameters:

• data – dictionary with all the necessary inputs

• exporter – exporter to call

• quiet – catch exception or not

• verbose – verbosity

• patch – apply patches

• optimization – optimization to do

• do_run – runs and compute discrepancies

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces

onnx_diagnostic.torch_models.test_helper.empty(value: Any) → bool

Tells if the value is empty.

onnx_diagnostic.torch_models.test_helper.get_inputs_for_task(task: str, config: Any | None = None) → Dict[str, Any]

Returns dummy inputs for a specific task.

Parameters:

• task – requested task

• config – returns dummy inputs for a specific config if available

Returns:

dummy inputs and dynamic shapes

onnx_diagnostic.torch_models.test_helper.split_args_kwargs(inputs: Any) → Tuple[Tuple[Any, ...], Dict[str, Any]]

Splits into args, kwargs.

onnx_diagnostic.torch_models.test_helper.validate_model(model_id: str, task: str | None = None, do_run: bool = False, exporter: str | None = None, do_same: bool = False, verbose: int = 0, dtype: str | dtype | None = None, device: str | device | None = None, trained: bool = False, optimization: str | None = None, quiet: bool = False, patch: bool = False, dump_folder: str | None = None) → Tuple[Dict[str, int | float | str], Dict[str, Any]]

Validates a model.

Parameters:

• model_id – model id to validate

• task – task used to generate the necessary inputs, can be left empty to use the default task for this model if it can be determined

• do_run – checks the model works with the defined inputs

• exporter – exporter the model using this exporter, available list: export-strict, export-nostrict, onnx

• do_same – checks the discrepancies of the exported model

• verbose – verbosity level

• dtype – uses this dtype to check the model

• device – do the verification on this device

• trained – use the trained model, not the untrained one

• optimization – optimization to apply to the exported model, depend on the the exporter

• quiet – if quiet, catches exception if any issue

• patch – applies patches before exporting

• dump_folder – dumps everything in a subfolder of this one

Returns:

two dictionaries, one with some metrics, another one with whatever the function produces