.torch_interpreter.diagnose

class experimental_experiment.torch_interpreter.diagnose.ModelDiagnoseOutput(name: str, model: Module, level: int = 0)

Contains inputs and outputs, diagnose results when tracing intermediate results. An instance of this class is produced by infer_shape_type_from_execution(). Example to_onnx, failures Phi-3.5-mini-instruct tells you more about how to use this class.

add_child(diag: ModelDiagnoseOutput)

Adds a submodule.

add_inputs(args: Tuple[Any, ...], kwargs: Dict[str, Any])

Stores used inputs. Makes a copy.

add_outputs(args: Tuple[Any, ...])

Stores returned outputs. Makes a copy.

property dot_name

Returns a kind of indented name.

property full_name

Returns a name and class name.

guess_dynamic_dimensions(*tensors) → Any

Infers the dynamic dimension from multiple shapes.

guess_dynamic_shape_object(*objs: Any, msg: Callable | None = None) → Any

Guesses the dynamic shapes for one argument.

guess_dynamic_shapes() → Any

Guesses the dynamic shapes for that module from two execution. If there is only one execution, then that would be static dimensions.

property module_name_type

Returns name and module type.

pretty_text(with_dynamic_shape: bool = False, with_shape: bool = True, with_min_max: bool = True, with_device: bool = True, with_inputs: bool = True) → str

Renders the outputs.

Parameters:

• with_dynamic_shape – show dynamic shapes

• with_shape – see experimental_experiment.helpers.string_type().

• with_min_max – see experimental_experiment.helpers.string_type().

• with_device – see experimental_experiment.helpers.string_type().

• with_inputs – show inputs and outputs shapes

Returns:

text

try_export(exporter: str = 'fx', exporter_kwargs: Dict[str, Any] | None = None, bypass_kwargs: Dict[str, Any] | None = None, verbose: int = 0, quiet: bool = True, discrepancies: bool = True, use_dynamic_shapes: bool | None = None, atol: float = 0.01, rtol: float = 0.1) → Any

Tries to export a model. If not possible, tries every child until it is possible. The fucntion stores the export and other results in the class itself.

Parameters:

• exporter – export way, ‘fx’ for torch.export.export(), ‘onnx_dynamo’ to call torch.onnx.export() (..., dynamo=True), ‘torch_script’ to call torch.onnx.export() (..., dynamo=False), ‘to_onnx’ to call experimental_experiment.torch_interpreter.to_onnx().

• exporter_kwargs – argument for the export function

• bypass_kwargs – argument for function bypass_export_some_errors

• verbose – verbosity, to see what the function is doing

• discrepancies – run the exported model to measure the discrepancies

• quiet – do not catch the first exception

• use_dynamic_shapes – use dynamic shapes

• atol – absolute tolerance

• rtol – relative tolerance

Returns:

result of the export function

See to_onnx, failures Phi-3.5-mini-instruct for an example.

experimental_experiment.torch_interpreter.diagnose.infer_shape_type_from_execution(model: Module, inputs: List[Tuple[Tuple[Any, ...], Dict[str, Any]]], verbose: int = 0) → ModelDiagnoseOutput

Runs a model, traces the intermediate output and infers dynamic shapes based on it.

Parameters:

• model – model

• inputs – list of input sets [(args, kwargs), (args, kwargs), ...] with different shapes (at least for the dynamic dimensions)

• verbose – verbosity

Returns:

see ModelDiagnoseOutput

See to_onnx, failures Phi-3.5-mini-instruct for an example.

experimental_experiment.torch_interpreter.diagnose.make_copy(obj: Any) → Any

Makes a copy of the objects.

experimental_experiment.torch_interpreter.diagnose.trace_forward_execution(model: Module, verbose: int = 0) → ModelDiagnoseOutput

Replaces all forward to store the inputs and outputs of the module and every submodules. See to_onnx, failures Phi-3.5-mini-instruct for an example.