experimental_experiment.torch_interpreter.oxs_dispatcher

class experimental_experiment.torch_interpreter.oxs_dispatcher.OxsDebugDispatcher(verbose: int = 0, raise_exc: bool = True)

Tries the fallback even if is not necessary to check it is working.

Parameters:

• verbose – verbosity

• raise_exc – fail or raise an exception

The class can be used the following way.

<<<

import torch
from experimental_experiment.torch_models.llama_helper import get_llama_model
from experimental_experiment.xbuilder import OptimizationOptions
from experimental_experiment.torch_interpreter import to_onnx
from experimental_experiment.torch_interpreter.oxs_dispatcher import (
    OxsDebugDispatcher,
)

with torch.no_grad():
    model, input_tensors = get_llama_model()
    input_tensors = input_tensors[0]

    to_onnx(
        model,
        input_tensors,
        input_names=[f"input{i}" for i in range(len(input_tensors))],
        options=OptimizationOptions(patterns=None),
        verbose=0,
        dispatcher=OxsDebugDispatcher(verbose=2, raise_exc=False),
    )

>>>

    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.embedding', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.arange', overload='start')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.full', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.triu', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.arange', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.reshape', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.gt', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul_', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.expand', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.clone', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.eq', overload='Scalar')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.masked_fill', overload='Scalar')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for 'aten_meth_expand_as' with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.expand', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.matmul', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.cat', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.cos', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.sin', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for 'aten_wrap_with_autocast' with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.pow', overload='Tensor_Scalar')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mean', overload='dim')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.rsqrt', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.view', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.view', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.view', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.unsqueeze', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.slice', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.slice', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.neg', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.cat', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.slice', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.slice', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.neg', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.cat', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.matmul', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.div', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.softmax', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.dropout', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.matmul', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.transpose', overload='int')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.contiguous', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.reshape', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.pow', overload='Tensor_Scalar')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mean', overload='dim')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.rsqrt', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.silu', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.linear', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.pow', overload='Tensor_Scalar')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mean', overload='dim')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.add', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.rsqrt', overload='default')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.to', overload='dtype')> with e='NoneType' object is not callable
    [OxsDebugDispatcher.fallback] fallback unverified for <OpOverload(op='aten.mul', overload='Tensor')> with e='NoneType' object is not callable

fallback(name: Any, fct: Callable | None, args: List[Any], kwargs: Dict[str, Any], builder: GraphBuilder) → Callable | None

The function is called after the function converting an aten function into ONNX. fct is this function. It can be changed and just set when mapping was found.

Parameters:

• name – object or str

• fct – function found so far

• args – known arguments coming from the graph module

• kwargs – known named arguments coming from the graph module

• builder – GraphBuilder

Returns:

callable

class experimental_experiment.torch_interpreter.oxs_dispatcher.OxsDispatcher(verbose: int = 0)

If DynamoInterpreter cannot find any converting function for a specific function, it tries to find an existing one in onnxscript. The converting function from onnxscript is run in trace only mode. The variable and functions op, Rank, IsScalar are replaced by op = OwsOpset(), op.Rank, op.Scalar. onnxscript may have multiple overloaded functions. Right now, it takes the first one.

Parameters:

verbose – verbose

fallback(name: Any, fct: Callable | None, args: List[Any], kwargs: Dict[str, Any], builder: GraphBuilder) → Callable | None

The function is called after the function converting an aten function into ONNX. fct is this function. It can be changed and just set when mapping was found.

Parameters:

• name – object or str

• fct – function found so far

• args – known arguments coming from the graph module

• kwargs – known named arguments coming from the graph module

• builder – GraphBuilder

Returns:

callable

property submodules: Dict[str, Callable]

Returns the submodules implementing torch functions.