yobx.xoptim.patterns.onnx_any#
- class yobx.xoptim.patterns.onnx_any.IdentityPattern(verbose: int = 0, priority: int = 0)[source]#
Replaces operator such as Div(X, 1), Mul(X, 1), Add(X, 0), Sub(X, 0), Transpose(X, [0, 1, 2, …]) by identity nodes. It looks into patterns involving the following operators:
<<<
from yobx.xoptim.patterns.onnx_any import IdentityPattern print(IdentityPattern.op_types)
>>>
{'And', 'Transpose', 'Or', 'Sub', 'BatchNormalization', 'Add', 'Slice', 'Mul', 'Div', 'Expand'}
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_x3(["x3 FLOAT(a, b, c)"]) Transpose_0[["Transpose(., perm=[0, 1, 2])"]] I_x3 -->|"FLOAT(a, b, c)"| Transpose_0 O_Y(["Y FLOAT(a, b, c)"]) Transpose_0 --> O_Y class I_x3,O_Y ioNode class Transpose_0 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_x3(["x3 FLOAT(a, b, c)"]) Identity_0[["Identity(.)"]] I_x3 -->|"FLOAT(a, b, c)"| Identity_0 O_Y(["Y FLOAT(a, b, c)"]) Identity_0 --> O_Y class I_x3,O_Y ioNode class Identity_0 opNode- apply(g: GraphBuilder, node: NodeProto) List[NodeProto][source]#
The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.
- Parameters:
nodes – nodes returned by method match, there are then removed
- Returns:
nodes to add to graph.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.NotNotPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#
Fuses Not + Not into Identity.
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X BOOL(a, 2)"]) Not_0[["Not(.)"]] Not_1[["Not(.)"]] I_X -->|"BOOL(a, 2)"| Not_0 Not_0 -->|"BOOL(a, 2)"| Not_1 O_Y(["Y BOOL(a, 2)"]) Not_1 --> O_Y class I_X,O_Y ioNode class Not_0,Not_1 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X BOOL(a, 2)"]) Identity_0[["Identity(.)"]] I_X -->|"BOOL(a, 2)"| Identity_0 O_Y(["Y BOOL(a, 2)"]) Identity_0 --> O_Y class I_X,O_Y ioNode class Identity_0 opNode- apply(g: GraphBuilder, not_before: NodeProto, not_after: NodeProto) List[NodeProto][source]#
The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.
- Parameters:
nodes – nodes returned by method match, there are then removed
- Returns:
nodes to add to graph.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.SameChildrenFromInputPattern(verbose: int = 0, priority: int = 0)[source]#
Checks there is no duplicated node doing the same than another one beside and taking a model input as input.
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_xy1(["xy1 FLOAT16(a, 2, 3, 4)"]) I_X(["X FLOAT(a, 2, 3, 4)"]) Cast_0[["Cast(., to=FLOAT16)"]] Cast_1[["Cast(., to=FLOAT16)"]] I_X -->|"FLOAT(a, 2, 3, 4)"| Cast_0 I_X -->|"FLOAT(a, 2, 3, 4)"| Cast_1 O_xy1(["xy1 FLOAT16(a, 2, 3, 4)"]) Cast_0 --> O_xy1 O_xy2(["xy2 FLOAT16(a, 2, 3, 4)"]) Cast_1 --> O_xy2 class I_xy1,I_X,O_xy1,O_xy2 ioNode class Cast_0,Cast_1 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_xy1(["xy1 FLOAT16(a, 2, 3, 4)"]) I_X(["X FLOAT(a, 2, 3, 4)"]) Cast_0[["Cast(., to=FLOAT16)"]] Identity_1[["Identity(.)"]] I_X -->|"FLOAT(a, 2, 3, 4)"| Cast_0 Cast_0 -->|"FLOAT16(a, 2, 3, 4)"| Identity_1 O_xy1(["xy1 FLOAT16(a, 2, 3, 4)"]) Cast_0 --> O_xy1 O_xy2(["xy2 FLOAT16(a, 2, 3, 4)"]) Identity_1 --> O_xy2 class I_xy1,I_X,O_xy1,O_xy2 ioNode class Cast_0,Identity_1 opNode- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.SameChildrenPattern(verbose: int = 0, priority: int = 0)[source]#
Checks there is no duplicated node doing the same than another one beside.
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_Y(["Y FLOAT(a, 1, 3, 4)"]) I_sh1(["sh1 INT64(4)"]) I_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0[["Expand(., .)"]] Expand_1[["Expand(., .)"]] I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_0 I_sh1 -->|"INT64(4)"| Expand_0 I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_1 I_sh1 -->|"INT64(4)"| Expand_1 O_y2(["y2 FLOAT(a, 2, 3, 4)"]) Expand_1 --> O_y2 O_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0 --> O_y1 class I_Y,I_sh1,I_y1,O_y2,O_y1 ioNode class Expand_0,Expand_1 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_Y(["Y FLOAT(a, 1, 3, 4)"]) I_sh1(["sh1 INT64(4)"]) I_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0[["Expand(., .)"]] Identity_1[["Identity(.)"]] I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_0 I_sh1 -->|"INT64(4)"| Expand_0 Expand_0 -->|"FLOAT(a, 2, 3, 4)"| Identity_1 O_y2(["y2 FLOAT(a, 2, 3, 4)"]) Identity_1 --> O_y2 O_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0 --> O_y1 class I_Y,I_sh1,I_y1,O_y2,O_y1 ioNode class Expand_0,Identity_1 opNode- apply(g: GraphBuilder, *nodes: NodeProto) List[NodeProto][source]#
The function receives pairs of nodes. We replace every odd node by an identity node.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.ShapeBasedIdentityPattern(verbose: int = 0, priority: int = 0)[source]#
If a slice leads to the same shape and the step is 1 then it is identity. In some cases, just known the same is enough to replace them.
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X FLOAT(a)"]) Shape_0[["Shape(., end=1, start=0)"]] Slice_1[["Slice(., [0], ., [0])"]] I_X -->|"FLOAT(a)"| Shape_0 I_X -->|"FLOAT(a)"| Slice_1 Shape_0 --> Slice_1 O_Y(["Y FLOAT(a)"]) Slice_1 --> O_Y class I_X,O_Y ioNode class Shape_0,Slice_1 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X FLOAT(a)"]) Identity_0[["Identity(.)"]] I_X -->|"FLOAT(a)"| Identity_0 O_Y(["Y FLOAT(a)"]) Identity_0 --> O_Y class I_X,O_Y ioNode class Identity_0 opNode- apply(g: GraphBuilder, node: NodeProto) List[NodeProto][source]#
The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.
- Parameters:
nodes – nodes returned by method match, there are then removed
- Returns:
nodes to add to graph.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.ShapeBasedSameChildrenPattern(verbose: int = 0, priority: int = 0)[source]#
Checks there is no duplicated node doing the same than another one beside.
yobx.xoptim.patterns.onnx_any.SameChildrenPatternchecks it is exactly the same. This one assumes it is exactly the same in some cases such expand (X, sh1) = expand(X, sh2) if the output shapes are the same.Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_y1(["y1 FLOAT(a, 2, 3, 4)"]) I_sh1(["sh1 INT64(4)"]) I_Y(["Y FLOAT(a, 1, 3, 4)"]) Expand_0[["Expand(., .)"]] Expand_1[["Expand(., .)"]] I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_0 I_sh1 -->|"INT64(4)"| Expand_0 I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_1 I_sh1 -->|"INT64(4)"| Expand_1 O_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0 --> O_y1 O_y2(["y2 FLOAT(a, 2, 3, 4)"]) Expand_1 --> O_y2 class I_y1,I_sh1,I_Y,O_y1,O_y2 ioNode class Expand_0,Expand_1 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_y1(["y1 FLOAT(a, 2, 3, 4)"]) I_sh1(["sh1 INT64(4)"]) I_Y(["Y FLOAT(a, 1, 3, 4)"]) Expand_0[["Expand(., .)"]] Identity_1[["Identity(.)"]] I_Y -->|"FLOAT(a, 1, 3, 4)"| Expand_0 I_sh1 -->|"INT64(4)"| Expand_0 Expand_0 -->|"FLOAT(a, 2, 3, 4)"| Identity_1 O_y1(["y1 FLOAT(a, 2, 3, 4)"]) Expand_0 --> O_y1 O_y2(["y2 FLOAT(a, 2, 3, 4)"]) Identity_1 --> O_y2 class I_y1,I_sh1,I_Y,O_y1,O_y2 ioNode class Expand_0,Identity_1 opNode- apply(g: GraphBuilder, *nodes: NodeProto) List[NodeProto][source]#
The function receives multiple nodes of the same type. We keep one and replace the other by an Identity node.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.
- class yobx.xoptim.patterns.onnx_any.SwapUnaryPattern(verbose: int = 0, priority: int = 0)[source]#
Tries to move computation nodes before any transpose or reshape. That works for unary operator or equivalent to that.
Model with nodes to be fused:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X FLOAT(a, b, c, d)"]) I_cst(["cst FLOAT(1)"]) Constant_0[["Constant() -#gt; cst"]] Transpose_1[["Transpose(., perm=[0, 2, 1, 3])"]] Mul_2[["Mul(., .)"]] I_X -->|"FLOAT(a, b, c, d)"| Transpose_1 Transpose_1 -->|"FLOAT(a, c, b, d)"| Mul_2 Constant_0 -->|"FLOAT(1)"| Mul_2 O_Y(["Y FLOAT(a, c, b, d)"]) Mul_2 --> O_Y class I_X,I_cst,O_Y ioNode class Constant_0 constNode class Transpose_1,Mul_2 opNodeOutcome of the fusion:
graph TD classDef ioNode fill:#dfd,stroke:#333,color:#333 classDef initNode fill:#cccc00,stroke:#333,color:#333 classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333 classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333 I_X(["X FLOAT(a, b, c, d)"]) I_cst(["cst FLOAT(1)"]) Mul_0[["Mul(., .)"]] Transpose_1[["Transpose(., perm=[0, 2, 1, 3])"]] I_X -->|"FLOAT(a, b, c, d)"| Mul_0 I_cst -->|"FLOAT(1)"| Mul_0 Mul_0 -->|"FLOAT(a, b, c, d)"| Transpose_1 O_Y(["Y FLOAT(a, c, b, d)"]) Transpose_1 --> O_Y class I_X,I_cst,O_Y ioNode class Mul_0,Transpose_1 opNode- apply(g: GraphBuilder, node: NodeProto, unary_node: NodeProto) List[NodeProto][source]#
The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.
- Parameters:
nodes – nodes returned by method match, there are then removed
- Returns:
nodes to add to graph.
- match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) MatchResult | None[source]#
Determines nodes around node which can be rewritten.
- Parameters:
g – is a
GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern
The method must not modify the graph. The method returns None if no match is found or an instance of class
MatchResult. It must contain:a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.