onnx_diagnostic.helpers.onnx_helper

class onnx_diagnostic.helpers.onnx_helper.NodeCoordinates(node: TensorProto | NodeProto | str, path: Tuple[Tuple[int, str, str], ...])

A way to localize a node, path is a tuple of three information, node index, node type, node name.

path_to_str() → str

Strings representing coordinates.

class onnx_diagnostic.helpers.onnx_helper.ResultFound(name: str, producer: NodeCoordinates | None, consumer: NodeCoordinates | None)

Class returned by enumerate_results().

onnx_diagnostic.helpers.onnx_helper.check_model_ort(onx: ModelProto, providers: str | List[Any] | None = None, dump_file: str | None = None) → onnxruntime.InferenceSession

Loads a model with onnxruntime.

Parameters:

• onx – ModelProto

• providers – list of providers, None fur CPU, cpu for CPU, cuda for CUDA

• dump_file – if not empty, dumps the model into this file if an error happened

Returns:

InferenceSession

onnx_diagnostic.helpers.onnx_helper.convert_endian(tensor: TensorProto) → None

Call to convert endianness of raw data in tensor.

Args:

tensor: TensorProto to be converted.

onnx_diagnostic.helpers.onnx_helper.dtype_to_tensor_dtype(dt: dtype | torch.dtype) → int

Converts a torch dtype or numpy dtype into a onnx element type.

Parameters:

to – dtype

Returns:

onnx type

onnx_diagnostic.helpers.onnx_helper.enumerate_results(proto: FunctionProto | GraphProto | ModelProto | Sequence[NodeProto], name: Set[str] | str, verbose: int = 0, coordinates: List[Tuple[int, str, str]] | None = None) → Iterator[ResultFound]

Iterates on all nodes, attributes to find where a name is used.

Parameters:

• proto – a proto

• name – name or names to find

• verbose – verbosity

• coordinates – coordinates of a node

Returns:

iterator on ResultFound

onnx_diagnostic.helpers.onnx_helper.extract_subset_of_nodes(model: ModelProto, name: str, node_index: int | None = None, cut_points: Set[str] | None = None) → List[NodeProto]

Extracts the minimal subgraphs which can produce the output name knowing cut_points.

Parameters:

• model – original model

• name – result name

• node_index – if the node index is known, otherwise searches for it

• cut_points – the known results or input name otherwise

Returns:

minimal list of nodes

onnx_diagnostic.helpers.onnx_helper.from_array_extended(tensor: ndarray | torch.Tensor, name: str | None = None) → TensorProto

Converts an array into a onnx.TensorProto.

Parameters:

• tensor – numpy array or torch tensor

• name – name

Returns:

TensorProto

onnx_diagnostic.helpers.onnx_helper.from_array_ml_dtypes(arr: ndarray | torch.Tensor, name: str | None = None) → TensorProto

Converts a numpy array to a tensor def assuming the dtype is defined in ml_dtypes.

Args:

arr: a numpy array. name: (optional) the name of the tensor.

Returns:

TensorProto: the converted tensor def.

onnx_diagnostic.helpers.onnx_helper.get_all_node_inputs(node: NodeProto) → Set[str]

Returns input and hidden inputs of a node. See get_hidden_inputs(). It excludes empty names.

onnx_diagnostic.helpers.onnx_helper.get_hidden_inputs(graph: GraphProto) → Set[str]

Returns the hidden inputs (inputs coming from an upper context) used by a subgraph. It excludes empty names.

onnx_diagnostic.helpers.onnx_helper.get_onnx_signature(model: ModelProto) → Tuple[Tuple[str, Any], ...]

Produces a tuple of tuples corresponding to the signatures.

Parameters:

model – model

Returns:

signature

onnx_diagnostic.helpers.onnx_helper.get_tensor_shape(obj: ValueInfoProto | TypeProto | TensorProto) → List[int | str | None] | None

Returns the shape if that makes sense for this object.

onnx_diagnostic.helpers.onnx_helper.iterator_initializer_constant(model: FunctionProto | GraphProto | ModelProto, use_numpy: bool = True, prefix: str = '') → Iterator[Tuple[str, torch.Tensor | ndarray]]

Iterates on iniatialiers and constant in an onnx model.

Parameters:

• model – model

• use_numpy – use numpy or pytorch

• prefix – for subgraph

Returns:

iterator

onnx_diagnostic.helpers.onnx_helper.make_submodel(nodes: List[NodeProto], ir_version: int, opset_imports: List[OperatorSetIdProto], output_names: List[str], type_rank_fn: Callable[[str], Tuple[int, int]]) → ModelProto

Creates a model with the given list of nodes. It computes the minimum list of inputs needed for this model. The function assumes the nodes are sorted. It does not handle yet subgraphs.

Parameters:

• nodes – list of nodes

• ir_version – ir version

• opset_imports – opset import

• output_names – desired outputs

• function – function returning the type and the rank of a result

Returns:

model proto

onnx_diagnostic.helpers.onnx_helper.np_dtype_to_tensor_dtype(dt: dtype) → int

Converts a numpy dtype into a onnx element type.

Parameters:

to – dtype

Returns:

onnx type

onnx_diagnostic.helpers.onnx_helper.onnx_dtype_name(itype: int, exc: bool = True) → str

Returns the ONNX name for a specific element type.

<<<

import onnx
from onnx_diagnostic.helpers.onnx_helper import onnx_dtype_name

itype = onnx.TensorProto.BFLOAT16
print(onnx_dtype_name(itype))
print(onnx_dtype_name(7))

>>>

    BFLOAT16
    INT64

onnx_diagnostic.helpers.onnx_helper.onnx_dtype_to_np_dtype(itype: int) → Any

Converts an onnx type into a to numpy dtype. That includes ml_dtypes dtypes.

Parameters:

to – onnx dtype

Returns:

numpy dtype

onnx_diagnostic.helpers.onnx_helper.onnx_find(onx: str | ModelProto, verbose: int = 0, watch: Set[str] | None = None) → List[NodeProto | TensorProto]

Looks for node producing or consuming some results.

Parameters:

• onx – model

• verbose – verbosity

• watch – names to search for

Returns:

list of nodes

onnx_diagnostic.helpers.onnx_helper.onnx_lighten(onx: str | ModelProto, verbose: int = 0) → Tuple[ModelProto, Dict[str, Dict[str, float]]]

Creates a model without big initializers but stores statistics into dictionaries. The function can be reversed with onnx_diagnostic.helpers.onnx_helper.onnx_unlighten(). The model is modified inplace.

Parameters:

• onx – model

• verbose – verbosity

Returns:

new model, statistics

onnx_diagnostic.helpers.onnx_helper.onnx_remove_node_unused(graph: GraphProto | FunctionProto, recursive=True) → GraphProto | FunctionProto

Removes unused nodes of the graph. An unused node is not involved in the output computation.

Parameters:

• onnx_model – onnx model

• recursive – looks into subgraphs

Returns:

new Graph

onnx_diagnostic.helpers.onnx_helper.onnx_unlighten(onx: str | ModelProto, stats: Dict[str, Dict[str, float]] | None = None, verbose: int = 0) → ModelProto

Function fixing the model produced by function onnx_diagnostic.helpers.onnx_helper.onnx_lighten(). The model is modified inplace.

Parameters:

• onx – model

• stats – statistics, can be None if onx is a file, then it loads the file <filename>.stats, it assumes it is json format

• verbose – verbosity

Returns:

new model, statistics

onnx_diagnostic.helpers.onnx_helper.pretty_onnx(onx: FunctionProto | GraphProto | ModelProto | ValueInfoProto | str, with_attributes: bool = False, highlight: Set[str] | None = None, shape_inference: bool = False) → str

Displays an onnx prot in a better way.

Parameters:

• with_attributes – displays attributes as well, if only a node is printed

• highlight – to highlight some names

• shape_inference – run shape inference before printing the model

Returns:

text

onnx_diagnostic.helpers.onnx_helper.select_model_inputs_outputs(model: ModelProto, outputs: List[str] | None = None, inputs: List[str] | None = None, infer_shapes: bool = True, overwrite: Dict[str, Any] | None = None, remove_unused: bool = True, verbose: int = 0)

Takes a model and changes its outputs.

Parameters:

• model – ONNX model

• inputs – new inputs, same ones if None

• outputs – new outputs, same ones if None

• infer_shapes – infer inputs and outputs shapes

• overwrite – overwrite type and shapes for inputs or outputs, overwrite is a dictionary {‘name’: (numpy dtype, shape)}

• remove_unused – remove unused nodes from the graph

• verbose – display information while converting

Returns:

modified model

The function removes unneeded nodes.

The following example shows how to change the inputs of model to bypass the first nodes. Shape inferences fails to determine the new inputs type. They need to be overwritten. verbose=1 shows the number of deleted nodes.

import onnx
from onnx_extended.tools.onnx_nodes import select_model_inputs_outputs

onx = onnx.load(path)
onx2 = select_model_inputs_outputs(
    onx, inputs=["a", "b"],
    infer_shapes=True, verbose=1,
    overwrite={'a': (numpy.int32, None), 'b': (numpy.int64, None)})
onnx.save(onx2, path2)

onnx_diagnostic.helpers.onnx_helper.shadowing_names(proto: FunctionProto | GraphProto | ModelProto | Sequence[NodeProto], verbose: int = 0, existing: Set[str] | None = None, shadow_context: Set[str] | None = None, post_shadow_context: Set[str] | None = None) → Tuple[Set[str], Set[str], Set[str]]

Returns the shadowing names, the names created in the main graph after they were created in a subgraphs and the names created by the nodes.

onnx_diagnostic.helpers.onnx_helper.tensor_dtype_to_np_dtype(tensor_dtype: int) → dtype

Converts a TensorProto’s data_type to corresponding numpy dtype. It can be used while making tensor.

Parameters:

tensor_dtype – TensorProto’s data_type

Returns:

numpy’s data_type

onnx_diagnostic.helpers.onnx_helper.tensor_statistics(tensor: ndarray | TensorProto) → Dict[str, float | str]

Produces statistics on a tensor.

Parameters:

tensor – tensor

Returns:

statistics

<<<

import pprint
import numpy as np
from onnx_diagnostic.helpers.onnx_helper import tensor_statistics

t = np.random.rand(40, 50).astype(np.float16)
pprint.pprint(tensor_statistics(t))

>>>

    {'>0.0': 2000,
     '>0.00010001659393310547': 2000,
     '>0.0010004043579101562': 1998,
     '>0.01000213623046875': 1974,
     '>0.0999755859375': 1788,
     '>0.5': 998,
     '>1.0': 0,
     '>1.0013580322265625e-05': 2000,
     '>1.0132789611816406e-06': 2000,
     '>1.1920928955078125e-07': 2000,
     '>1.9599609375': 0,
     '>10.0': 0,
     '>100.0': 0,
     '>1000.0': 0,
     '>10000.0': 0,
     'itype': 10,
     'max': 0.9990234375,
     'mean': 0.49658203125,
     'min': 0.0005383491516113281,
     'nnan': 0.0,
     'numel': 2000,
     'q0.1': 0.09334716796875,
     'q0.2': 0.1926025390625,
     'q0.3': 0.294677734375,
     'q0.4': 0.39672851562500006,
     'q0.5': 0.4984130859375,
     'q0.6': 0.6001953124999998,
     'q0.7': 0.6943359375,
     'q0.8': 0.79697265625,
     'q0.9': 0.8974609375,
     'shape': '40x50',
     'size': 4000,
     'std': 0.289794921875,
     'stype': 'FLOAT16'}

onnx_diagnostic.helpers.onnx_helper.to_array_extended(proto: TensorProto) → ndarray | torch.Tensor

Converts onnx.TensorProto into a numpy array.

onnx_diagnostic.helpers.onnx_helper.type_info(itype: int, att: str)

Returns the minimum or maximum value for a type.

Parameters:

• itype – onnx type

• att – ‘min’ or ‘max’

Returns:

value