experimental_experiment.torch_interpreter.eval¶

modules

experimental_experiment.torch_interpreter.eval.model_cases

experimental_experiment.torch_interpreter.eval.discover()[source]¶

Discovers all model cases used to evaluate an exporter.

<<<

import pprint
from experimental_experiment.torch_interpreter.eval import discover

pprint.pprint(discover())

>>>

    {'AtenAsStrided': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenAsStrided'>,
     'AtenInterpolate': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenInterpolate'>,
     'AtenNonZero': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenNonZero'>,
     'AtenNonZeroTuple': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenNonZeroTuple'>,
     'AtenRollPos': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenRollPos'>,
     'AtenRollRelu': <class 'experimental_experiment.torch_interpreter.eval.model_cases.AtenRollRelu'>,
     'BuildInIsInstance': <class 'experimental_experiment.torch_interpreter.eval.model_cases.BuildInIsInstance'>,
     'BuildInLen': <class 'experimental_experiment.torch_interpreter.eval.model_cases.BuildInLen'>,
     'ComplexPolar': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ComplexPolar'>,
     'ControlFlowCond': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowCond'>,
     'ControlFlowCond2Inputs': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowCond2Inputs'>,
     'ControlFlowCond2Outputs': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowCond2Outputs'>,
     'ControlFlowCondConstant': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowCondConstant'>,
     'ControlFlowCondNestedModule': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowCondNestedModule'>,
     'ControlFlowNestCond': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowNestCond'>,
     'ControlFlowScan': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowScan'>,
     'ControlFlowScan2Carried': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowScan2Carried'>,
     'ControlFlowScanCDist': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowScanCDist'>,
     'ControlFlowScanCDist2': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowScanCDist2'>,
     'ControlFlowScanCDistXY': <class 'experimental_experiment.torch_interpreter.eval.model_cases.ControlFlowScanCDistXY'>,
     'CropLastDimensionWithTensorContent': <class 'experimental_experiment.torch_interpreter.eval.model_cases.CropLastDimensionWithTensorContent'>,
     'CropLastDimensionWithTensorShape': <class 'experimental_experiment.torch_interpreter.eval.model_cases.CropLastDimensionWithTensorShape'>,
     'InplaceAdd': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceAdd'>,
     'InplaceAdd_': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceAdd_'>,
     'InplaceAdd_Mul': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceAdd_Mul'>,
     'InplaceCloneAdd_': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceCloneAdd_'>,
     'InplaceSetItemEllipsis_1': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemEllipsis_1'>,
     'InplaceSetItemEllipsis_2': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemEllipsis_2'>,
     'InplaceSetItemMask': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemMask'>,
     'InplaceSetItemSquare': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemSquare'>,
     'InplaceSetItemSquareAdd': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemSquareAdd'>,
     'InplaceSetItemSquareAdd2': <class 'experimental_experiment.torch_interpreter.eval.model_cases.InplaceSetItemSquareAdd2'>,
     'SignatureFloat1': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureFloat1'>,
     'SignatureInt1': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureInt1'>,
     'SignatureInt2': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureInt2'>,
     'SignatureListFixedLength': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureListFixedLength'>,
     'SignatureListVariableLength': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureListVariableLength'>,
     'SignatureShapeAsIndex': <class 'experimental_experiment.torch_interpreter.eval.model_cases.SignatureShapeAsIndex'>,
     'TypeBFloat16': <class 'experimental_experiment.torch_interpreter.eval.model_cases.TypeBFloat16'>}

experimental_experiment.torch_interpreter.eval.evaluation(exporters: Tuple[str] = ('export-strict', 'export-strict-dec', 'export-nostrict', 'export-nostrict-dec', 'export-tracing', 'custom-strict', 'custom-strict-dec', 'custom-nostrict', 'custom-nostrict-dec', 'custom-tracing'), dynamic: Tuple[bool] = (False, True), cases: Dict[str, type] | None = None, verbose: int = 0, quiet: bool = True) → List[Dict[str, Any]][source]¶

Evaluates exporter for a list of cases.

Parameters:

exporters – exporters to evaluate
dynamic – evaluate static shape and dynamic shapes
cases – model cases to evaluate
verbose – verbosity
quiet – catch exception

Returns:

results, list of dictionaries

experimental_experiment.torch_interpreter.eval.run_exporter(exporter: str, cls_model: type, dynamic: bool = False, quiet: bool = False, verbose: int = 0) → Dict[str, Any][source]¶

Runs an exporter and returns whether it fails or not.

Parameters:

exporter – exporter
cls_model – model class to create
inputs – list of inputs to try
dynamic – use dynamic shape or not
quiet – raise exception or not
verbose – verbosity

Returns:

results