graph_builder

GraphBuilder

class experimental_experiment.xbuilder.GraphBuilder(target_opset_or_existing_proto: int | Dict[str, int] | ModelProto | FunctionProto, input_names: Sequence[str] | None = None, as_function: bool = False, optimization_options: OptimizationOptions | None = None, args: List[Any] | None = None, ir_version: int | None = None, verbose: int = 0, infer_shapes: bool = False, raise_list: Set[str] | None = None, dynamic_shapes: Dict[str, Any] | Tuple[Any] | None = None)

Simplifies the creation of a model. Important attributes:

• input_names: List[str]: list of input names

• as_function: bool: the model must be exported as a function or as a model

• optimization_options: OptimizationOptions:

• nodes: List[NodeProto]: list of nodes

• initializers_dict: Dict[str, Any]: initializers

• inputs: List[ValueInfoTensorProto]: inputs

• outputs: List[ValueInfoTensorProto]: outputs

• ir_version: int: ir version

• opsets: Dict[str, int]: declared opsets

• input_args: List[T]: input tensors when the class is used to convert an existing model

• functions: List[FunctionProto]: list of functions to add to the model

• value_info: List[ValueInfoProto]: value info of the original model

• dynamic_shapes: Union[Dict[str, Any], Tuple[Any]]]: dynamic_shapes informations

Computed attributes:

• _unique_names: used to create unused result names

• _unique_node_names: used to create unused node names

• _known_names: set of existing results names

• _known_shapes: Dict[str, DYNAMIC_SHAPE]: declared shapes

• _known_types: Dict[str, int]: declared element types

• _known_value_shape: Dict[str, Any]: if a result is a shape or not (for example the output of operator Shape)

• _known_ranks: Dict[str, int]: declared ranks

• _known_sequences: Dict[str, Dict[str, Any]]: known sequences

• constants_node_: Dict[bytes, NodeProto]: constant node

• constants_alias_: Dict[str, str]: alias for constant

• constants_: Dict[str, Any]: constant values

• constants_computed_: Dict[str, Any]: computed constant values

• dynamic_objects: Dict[str, torch.SymInt]: list of dynamic dimension

• dynamic_objects_rev: Dict[str, str]: reverse dictionary to fasten lookups

• _cache_shape: Dict[key,str]: cache concatenation of shapes

• _values: Dict[key,str]: cache initializer value to merge those which are equal

• _dynamic_alias: Dict[str,str]: used when the user gives a different

  name to the dynamic shapes

Debugging attributes:

• _raise_list: Set[str]: the builder stop if a result falls in that list (debugging tool)

add_constant_node(node: NodeProto) → bytes | None

Adds a constant node. Any constant equivalent to this one will be fused. self.optimization_options.constant_fusing must be True.

add_domain(domain: str, version: int = 1)

Adds a domain to the list of supported ones. Checks the version is the same if it exists.

constant_folding(convert_into_initializer: bool = True) → int

Folds all constants. Constants are marked during the creation of the graph. There is no need to propagate this information.

Parameters:

convert_into_initializer – moves the constant as an initializer, otherwise, just evaluates it

Returns:

number of removed nodes

do_not_remove(node: NodeProto) → bool

Tells if a node should be removed or not.

elem_size(elem_type: int) → int

Returns the size in byte of the an element of this size.

empty_copy(as_function: bool = False, constant_size: int = 16777216) → GraphBuilder

Creates an empty copy but with the same opsets.

get_attribute(node: NodeProto, att_name: str, exc: bool = True) → AttributeProto | None

Returns an attribute for a node.

get_attributes_with_default(node: NodeProto, **default_values) → Dict[str, Any]

Returns int or float attributes. If missing, the default value is returned.

Parameters:

• node – node

• default_values – default values

get_constant(name: str, exc: bool = True, computed_value: bool = False, as_shape: bool = False) → ndarray | NodeProto

The method returns the constant name. It is a tensor (numpy array) or a NodeProto which must be evaluated. If computed_value is True, the NodeProto is evaluated wuth the ReferenceEvaluator.

Parameters:

• name – constant name

• exc – raise an exception if anything is impossible to do

• computed_value – compute the value if not a constant

• as_shape – returns a tuple for a shape

Returns:

value

get_constant_or_attribute(node: NodeProto, input_index: int, att_name: str) → Any

Tells if an input is a constant or returns true if in an older opset, it was named as an attribute.

get_debug_msg() → str

Returns a string providing as much information as possible to help the developper understand why a conversion failed.

get_is_dimension(name: str, elem_type: int | None = None, shape: Tuple[int, ...] | None = None) → bool

Tells if a result is a dynamic dimension or not.

get_opset(domain: str) → int

Returns the opset version for a specific domain.

Parameters:

domain – domain name

Returns:

version

get_rank(name: str) → int

Returns the rank of a result.

get_sequence(name: str) → Dict[str, Any]

Returns sequence information

get_shape(name: str) → int

Returns the shape of a result.

get_type(name: str) → int

Returns the type of a result.

has_dynamic_object(name: str) → bool

Tells if a result is a dynamic object, torch.SymInt for torch.

has_name(name: str) → bool

Tells if a result exists.

has_rank(name: str) → bool

Tells if a result has a rank.

has_shape(name: str, full=False) → bool

Tells if a result has a shape.

has_type(name: str) → bool

Tells if a result has a type. This should be always true.

insert_and_remove_nodes(insert_at: int | None, new_nodes: List[NodeProto], removed: List[int], opsets: Dict[str, int] | None = None) → List[NodeProto]

Inserts new nodes and removes others.

Parameters:

• insert_at – insert the new nodes at this position, if empty, the function guesses where to add them

• new_nodes – list of nodes to insert

• removed – list of nodes to removed (based on their positions)

• opsets – opsets used

Returns:

list of removed nodes

io_names()

Returns the list of inputs, output for nodes.

is_constant(name: str) → bool

Tells if a result is a constant.

is_constant_or_attribute(node: NodeProto, input_index: int, att_name: str) → bool

Tells if an input is a constant or returns true if in an older opset, it was named as an attribute.

is_exact_same_constant(node: NodeProto) → NodeProto | None

Adds a constant node. Any constant equivalent to this one will be fused. self.optimization_options.constant_fusing must be True.

is_sequence(name: str) → bool

Tells if a result is a sequence.

property main_opset

Returns the opset for the main domain (assuming it is used).

make_dynamic_object(name: str, value: Any, shape_as_input: bool = False) → str

Creates a dynamic shapes.

Parameters:

• name – name

• value – value

• shape_as_input – adds the name to the list of the inputs of the onnx model

Returns:

the name

make_initializer(name: str, value: Any, external: bool = False, msg: str = '') → str

Adds an initializer to the graph. The function detects duplicated small containers, only if they are integers. Other type might be used as weights. Even similar, they could change after training.

Parameters:

• name – name, if empty (“”), a unique names is given, if not empty, it is more like a prefix, the method might change it to make it unique

• value – value (TensorProto)

• external – external initializer or not (not stored in the graph model)

• msg – added to the error message if something goes wrong

Returns:

name of the initializer

make_key(value: Any) → Tuple[str | int, ...] | None

Builds a key identifying a value. Returns None if it is none possible.

make_node(op_type: str, inputs: str | List[str], outputs: int | List[str] | str = 1, domain: str = '', attributes: List[AttributeProto] | None = None, check: bool | None = None, name: str | None = None, sts: Dict[str, Any] | None = None, do_not_remove: bool = False, **kwargs) → str | List[str]

Adds a node in the graph.

Parameters:

• op_type – operator type

• inputs – input names

• outputs – output names, may be None, in that case, the builder chooses them for the user

• domain – domain

• attributes – list of attributes to add as AttributeProto

• check – do some verification

• name – node name

• sts – if not specified, tries to set the shape and the type of the new results aftr the node is added, it is not possible for every node, there is no tool which determines the output shape of just one node

• do_not_remove – prevent this node from being removed

• kwargs – additional attributes to add the node

Returns:

output names

make_nodes(builder: GraphBuilder, input_names: List[str], output_names: List[str], prefix: str = '') → str | List[str]

Appends all nodes and initializers from another builder. Handles the renaming of results. The content stored in ‘builder’ is modified inplace to avoid copying.

Parameters:

• builder – other builder

• input_names – input names

• output_names – output names

• prefix – prefix all name from this builder

Returns:

output names

make_shape_from_results(shape: Tuple[int | torch.SymInt | str, ...], name='') → str

Creates a shape coming from intermediate results.

make_tensor_input(name: str, elem_type: Any, shape: Tuple[int, ...], is_dimension: bool) → str

Adds a tensor input to the onnx graph.

Parameters:

• name – name

• elem_type – element type

• shape – shape

• is_dimension – torch is using torch.SymInt to add a dynamic input to the graph

Returns:

input name

make_tensor_output(name: str | List[str], elem_type: int | None = None, shape: Tuple[int, ...] | None = None, indexed: bool = True, is_dimension: bool | None = None) → str | List[str]

Adds a tensor output to the onnx graph.

Parameters:

• name – name

• elem_type – element type

• shape – shape

• indexed – the name must be indexed?

• is_dimension – torch is using torch.SymInt to add a dynamic input to the graph

Returns:

output name

optimize() → List[Dict[str, Any]]

Optimizes a graph. Returns the list of applied processed.

optimize_with_patterns() → List[Dict[str, Any]]

Optimizes this graph with patterns.

parse_dimension_expression(expr: str, exc: bool = True) → Expression

Parses an expression involving dimension.

Parameters:

• expr – expr

• exc – raises an exception if it fails

Returns:

an expression or None if exc is False and the parsing failed

rank(name: str) → int

Shortcut to get_rank().

remove_identity_nodes() → Tuple[int, int]

Removes identity nodes. Returns the number of removed nodes and the number of added nodes.

Note

onnxruntime does not handle well when it is executing from domain ‘org.pytorch.aten’ (ATen for example) which outputs results on CPU where the expected output is on CUDA. An identity node must be kept or inserted in that case. In that particular case, a node can be marked so that it does not get deleted: its name must start with '_DONOTREMOVE_'.

remove_unused() → int

Simple function to remove unused nodes. It does not look into subgraphs and assumes there is none. Everything is done in one pass. Returns the number of removed nodes.

select_outputs(output_names: List[str])

Selects new outputs. The type is assumed to be unknown. The method only wipes out the outputs to replace them by others. It assumes the unused nodes are removed afterwards.

Parameters:

output_names – new outputs

set_name(name: str)

Adds a name to the list of known names.

set_rank(name: str, value: int)

Sets the rank for a result.

Parameters:

• name – result name

• value – rank

set_sequence(name: str, dtype: int, shapes: Tuple[int | torch.SymInt | str, ...] | None = None, ranks: Tuple[int, ...] | None = None, unknown: bool = False)

Defines a result as a sequence.

set_shape(name: str, shape: Tuple[int | torch.SymInt | str, ...], set_rank: bool = True, set_if_more_precise: bool = False, exc: bool = False)

Sets the shape for a result. It is exists, it checks the new shape is equal to the existing one.

Parameters:

• name – result name

• shape – shape

• set_rank – set the rank as well

• set_if_more_precise – change the shape if it is more precise

• exc – raise an exception if inconsistency

set_type(name: str, dtype: int)

Sets the shape for a result. It is exists, it checks the new shape is equal to the existing one.

Parameters:

• name – name

• dtype – element type (an integer, ONNX)

set_value_shape(name: str, value: Any)

Sets the value for a shape result.

Parameters:

• name – name

• value – it cannot be empty

to_onnx(as_function: bool = False, optimize: bool = True, large_model: bool = False, external_threshold: int = 1024) → FunctionProto | ModelProto | TorchModelContainer

Conversion to onnx. Only then the initializer are converted into TensorProto.

Parameters:

• as_function – converts the graph as a FunctionProto or a ModelProto

• optimize – disable or enable the optimization, the optimization are set when the class constructor is called

• large_model – if True returns a onnx.model_container.ModelContainer, it lets the user to decide later if the weights should be part of the model or saved as external weights

• external_threshold – if large_model is True, every tensor above this limit is stored as external

Returns:

the proto

value_as_shape(name: str) → bool

Returns the value of a result if it is a shape.

verify_dynamic_shape(shape: Any, name: str | None = None, add: bool = True) → Tuple[int | torch.SymInt | str, ...]

The implementation of this method should be revisited.

OptimizationOptions

class experimental_experiment.xbuilder.OptimizationOptions(remove_unused: bool = True, constant_folding: bool = False, constant_size: int = 1024, constant_fusing: bool = True, remove_identity: bool = True, patterns: str | List[PatternOptimization] = 'default', max_iter: int = -1, recursive: bool = False, stop_after: int = -1, verbose: int = 0, verifies: bool = False, dump_applied_patterns: str | None = None, processor: str = 'CPU', order: OrderAlgorithm | None = None)

Defines all the optimization to apply.

Parameters:

• remove_unused – remove all unused nodes, this must be true if pattern optimization is enabled

• constant_folding – folds constant as much as possible

• constant_size – all node Constant above this threshold should be defined as initializer

• remove_identity – remove identity nodes

• patterns – list of pattern optimization to apply to the graph, it looks a a specific subsequence of nodes in a graph and do some replacements, ‘default’ means a default list of optimization patterns are applied

• constant_fusing – similar node Constant and ConstantOfShape are used, this options avoids creating new nodes when they are the same

• max_iter – maximum number of iteration when doing pattern optimizations, -1 to let it undefined

• recursive – optimizes subgraphs and functions as well

• stop_after – for investigation, stop_after this number of applies patterns, -1 to never stop

• verbose – verbosity level (for pattern optimization)

• verifiers – run verifications to ensure the model is correct everytime it is modifies, it is mostly to find bugs, it is very slow

• dump_applied_patterns – dump applied patterns in a folder, the users can check every pattern dumped as a FunctionProto

• processor – optimization should be made for this processor or this list of processors (comma separated value)

• order – order algorithm to apply

Opset

class experimental_experiment.xbuilder.graph_builder.Opset(builder: GraphBuilder, allow_unknown: bool = False)

Makes it easier to write onnx graph. The method name is the node type.

Parameters:

• graph_builder – the builder

• allow_unknown – allows unknown operators, otherwise, fails this class does not the expected number of outputs