yobx.xoptim.patterns.onnx_matmul — yet-another-onnx-builder 0.1.0 documentation

class yobx.xoptim.patterns.onnx_matmul.GemmTransposePattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces Gemm (., constant) by Gemm(., constant’, transB=1)

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(3, 2)"])
    I_X(["X FLOAT(2, 3)"])

    Constant_0[["Constant() -#gt; B"]]
    Gemm_1[["Gemm(., .)"]]

    I_X -->|"FLOAT(2, 3)"| Gemm_1
    Constant_0 -->|"FLOAT(3, 2)"| Gemm_1

    O_Z(["Z FLOAT(2, 2)"])
    Gemm_1 --> O_Z

    class I_B,I_X,O_Z ioNode
    class Constant_0 constNode
    class Gemm_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(3, 2)"])
    I_X(["X FLOAT(2, 3)"])

    Transpose_0[["Transpose(., perm=[1, 0])"]]
    Gemm_1[["Gemm(., .)"]]

    I_B -->|"FLOAT(3, 2)"| Transpose_0
    I_X -->|"FLOAT(2, 3)"| Gemm_1
    Transpose_0 -->|"FLOAT(2, 3)"| Gemm_1

    O_Z(["Z FLOAT(2, 2)"])
    Gemm_1 --> O_Z

    class I_B,I_X,O_Z ioNode
    class Transpose_0,Gemm_1 opNode

apply(g: GraphBuilder, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.MatMulAddPattern(verbose: int = 0, priority: int = 3, allow_reshape: bool = False)[source]#

Replaces the sequence MatMul, Add into Gemm. By default, no reshape is allowed this happens only it is two dimensions.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(a, b, d)"])
    I_X1(["X1 FLOAT(a, b, 3)"])
    I_X2(["X2 FLOAT(3, d)"])

    MatMul_0[["MatMul(., .)"]]
    Add_1[["Add(., .)"]]

    I_X1 -->|"FLOAT(a, b, 3)"| MatMul_0
    I_X2 -->|"FLOAT(3, d)"| MatMul_0
    MatMul_0 -->|"FLOAT(a, b, d)"| Add_1
    I_B -->|"FLOAT(a, b, d)"| Add_1

    O_Z(["Z FLOAT(a, b, d)"])
    Add_1 --> O_Z

    class I_B,I_X1,I_X2,O_Z ioNode
    class MatMul_0,Add_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(a, b, d)"])
    I_X1(["X1 FLOAT(a, b, 3)"])
    I_X2(["X2 FLOAT(3, d)"])

    Reshape_0[["Reshape(., [-1, 3])"]]
    Shape_1[["Shape(., start=-1)"]]
    Concat_2[["Concat([-1], ., axis=0)"]]
    Reshape_3[["Reshape(., .)"]]
    Shape_4[["Shape(., end=-1, start=0)"]]
    Concat_5[["Concat(., [-1], axis=0)"]]
    Gemm_6[["Gemm(., ., .)"]]
    Reshape_7[["Reshape(., .)"]]

    I_X1 -->|"FLOAT(a, b, 3)"| Reshape_0
    I_B -->|"FLOAT(a, b, d)"| Shape_1
    Shape_1 -->|"INT64(1)"| Concat_2
    I_B -->|"FLOAT(a, b, d)"| Reshape_3
    Concat_2 -->|"INT64(2)"| Reshape_3
    I_X1 -->|"FLOAT(a, b, 3)"| Shape_4
    Shape_4 -->|"INT64(2)"| Concat_5
    Reshape_0 -->|"FLOAT(a*b, 3)"| Gemm_6
    I_X2 -->|"FLOAT(3, d)"| Gemm_6
    Reshape_3 -->|"FLOAT(a*b, d)"| Gemm_6
    Gemm_6 -->|"FLOAT(a*b, d)"| Reshape_7
    Concat_5 -->|"INT64(3)"| Reshape_7

    O_Z(["Z FLOAT(a, b, d)"])
    Reshape_7 --> O_Z

    class I_B,I_X1,I_X2,O_Z ioNode
    class Reshape_0,Shape_1,Concat_2,Reshape_3,Shape_4,Concat_5,Gemm_6,Reshape_7 opNode

apply(g: GraphBuilder, matmul_node: NodeProto, add_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.MatMulReshape2Of3Pattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the reshapes around a matmul It can be 3 or 2 out of 3. It is similar to yobx.xoptim.patterns.onnx_reshape.Reshape2Of3Pattern.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_div_5(["div_5 FLOAT(13, 4, 7, 7)"])
    I_transpose_23(["transpose_23 FLOAT(52, 7, 8)"])
    I_init7_s4_13_4_7_8(["init7_s4_13_4_7_8 INT64(4)"])

    Constant_0[["Constant() -#gt; init7_s4_13_4_7_8"]]
    Reshape_1[["Reshape(., [52, 7, 7])"]]
    MatMul_2[["MatMul(., .)"]]
    Reshape_3[["Reshape(., .)"]]

    I_div_5 -->|"FLOAT(13, 4, 7, 7)"| Reshape_1
    Reshape_1 -->|"FLOAT(52, 7, 7)"| MatMul_2
    I_transpose_23 -->|"FLOAT(52, 7, 8)"| MatMul_2
    MatMul_2 -->|"FLOAT(52, 7, 8)"| Reshape_3
    Constant_0 -->|"INT64(4)"| Reshape_3

    O_view_85(["view_85 FLOAT(13, 4, 7, 8)"])
    Reshape_3 --> O_view_85

    class I_div_5,I_transpose_23,I_init7_s4_13_4_7_8,O_view_85 ioNode
    class Constant_0 constNode
    class Reshape_1,MatMul_2,Reshape_3 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_div_5(["div_5 FLOAT(13, 4, 7, 7)"])
    I_transpose_23(["transpose_23 FLOAT(52, 7, 8)"])
    I_init7_s4_13_4_7_8(["init7_s4_13_4_7_8 INT64(4)"])

    Reshape_0[["Reshape(., .)"]]
    MatMul_1[["MatMul(., .)"]]

    I_transpose_23 -->|"FLOAT(52, 7, 8)"| Reshape_0
    I_init7_s4_13_4_7_8 -->|"INT64(4)"| Reshape_0
    I_div_5 -->|"FLOAT(13, 4, 7, 7)"| MatMul_1
    Reshape_0 --> MatMul_1

    O_view_85(["view_85 FLOAT(13, 4, 7, 8)"])
    MatMul_1 --> O_view_85

    class I_div_5,I_transpose_23,I_init7_s4_13_4_7_8,O_view_85 ioNode
    class Reshape_0,MatMul_1 opNode

apply(g: GraphBuilder, node_left: NodeProto | None, node_right: NodeProto | None, node: NodeProto, next_node: NodeProto | None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.MulMulMatMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces MatMul(a*c, b*d) where c and d are constant scalar by MatMul(a,b) * (c,d).

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(16, 64)"])
    I_X(["X FLOAT(32, 16)"])

    Mul_0[["Mul(., [0.4])"]]
    Mul_1[["Mul([0.6], .)"]]
    MatMul_2[["MatMul(., .)"]]

    I_X -->|"FLOAT(32, 16)"| Mul_0
    I_Y -->|"FLOAT(16, 64)"| Mul_1
    Mul_0 -->|"FLOAT(32, 16)"| MatMul_2
    Mul_1 -->|"FLOAT(16, 64)"| MatMul_2

    O_z(["z FLOAT(32, 64)"])
    MatMul_2 --> O_z

    class I_Y,I_X,O_z ioNode
    class Mul_0,Mul_1,MatMul_2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(16, 64)"])
    I_X(["X FLOAT(32, 16)"])

    MatMul_0[["MatMul(., .)"]]
    Mul_1[["Mul(., [0.24000001])"]]

    I_X -->|"FLOAT(32, 16)"| MatMul_0
    I_Y -->|"FLOAT(16, 64)"| MatMul_0
    MatMul_0 -->|"FLOAT(32, 64)"| Mul_1

    O_z(["z FLOAT(32, 64)"])
    Mul_1 --> O_z

    class I_Y,I_X,O_z ioNode
    class MatMul_0,Mul_1 opNode

apply(g: GraphBuilder, mul1: NodeProto, mul2: NodeProto, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.ReshapeMatMulReshapePattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence Reshape, Matmul, Reshape by Matmul.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_xu2(["xu2 FLOAT(1, 1, 32, 128)"])
    I_Y(["Y FLOAT(3, 5, 128, 64)"])

    Reshape_0[["Reshape(., [1, 32, 128])"]]
    Reshape_1[["Reshape(., [15, 128, 64])"]]
    MatMul_2[["MatMul(., .)"]]
    Reshape_3[["Reshape(., [3, 5, 32, 64])"]]

    I_xu2 -->|"FLOAT(1, 1, 32, 128)"| Reshape_0
    I_Y -->|"FLOAT(3, 5, 128, 64)"| Reshape_1
    Reshape_0 -->|"FLOAT(1, 32, 128)"| MatMul_2
    Reshape_1 -->|"FLOAT(15, 128, 64)"| MatMul_2
    MatMul_2 -->|"FLOAT(15, 32, 64)"| Reshape_3

    O_Z(["Z FLOAT(3, 5, 32, 64)"])
    Reshape_3 --> O_Z

    class I_xu2,I_Y,O_Z ioNode
    class Reshape_0,Reshape_1,MatMul_2,Reshape_3 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_xu2(["xu2 FLOAT(1, 1, 32, 128)"])
    I_Y(["Y FLOAT(3, 5, 128, 64)"])

    MatMul_0[["MatMul(., .)"]]

    I_xu2 -->|"FLOAT(1, 1, 32, 128)"| MatMul_0
    I_Y -->|"FLOAT(3, 5, 128, 64)"| MatMul_0

    O_Z(["Z FLOAT(3, 5, 32, 64)"])
    MatMul_0 --> O_Z

    class I_xu2,I_Y,O_Z ioNode
    class MatMul_0 opNode

apply(g: GraphBuilder, node_before_left: NodeProto, node_before_right: NodeProto, node: NodeProto, next_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.ShapeBasedMatMulToMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

MatMul can be replaced by Mul with broadcast. It makes it easier to detect optimization pattern with Expand operators.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(a, 1, c)"])
    I_X(["X FLOAT(a, b, 1)"])

    MatMul_0[["MatMul(., .)"]]
    Transpose_1[["Transpose(., perm=[0, 2, 1])"]]

    I_X -->|"FLOAT(a, b, 1)"| MatMul_0
    I_Y -->|"FLOAT(a, 1, c)"| MatMul_0
    MatMul_0 -->|"FLOAT(a, b, c)"| Transpose_1

    O_Z(["Z FLOAT(a, c, b)"])
    Transpose_1 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class MatMul_0,Transpose_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(a, 1, c)"])
    I_X(["X FLOAT(a, b, 1)"])

    Reshape_0[["Reshape(., [0, 1, -1])"]]
    Reshape_1[["Reshape(., [0, -1, 1])"]]
    Mul_2[["Mul(., .)"]]

    I_X -->|"FLOAT(a, b, 1)"| Reshape_0
    I_Y -->|"FLOAT(a, 1, c)"| Reshape_1
    Reshape_0 -->|"FLOAT(a, 1, b)"| Mul_2
    Reshape_1 -->|"FLOAT(a, c, 1)"| Mul_2

    O_Z(["Z FLOAT(a, c, b)"])
    Mul_2 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class Reshape_0,Reshape_1,Mul_2 opNode

apply(g: GraphBuilder, mm_node: NodeProto, transpose: NodeProto | None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult], left_first: bool = True) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.SwitchReshapeActivationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Switches Gelu and Reshape after a Gemm or a MatMul. Gelu can also be Gelu, Exp, Elu, Relu, Tan, Tanh, Cos, Cosh, Sin, Sinh, Erf, LeakyRelu, PRelu, Selu, Softmax, Softplus. Reshape can also be Transpose.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(3, 2, 5, 6)"])
    I_X(["X FLOAT(3, 2, 6, 5)"])

    MatMul_0[["MatMul(., .)"]]
    Transpose_1[["Transpose(., perm=[0, 2, 1, 3])"]]
    Relu_2[["Relu(.)"]]

    I_X -->|"FLOAT(3, 2, 6, 5)"| MatMul_0
    I_Y -->|"FLOAT(3, 2, 5, 6)"| MatMul_0
    MatMul_0 -->|"FLOAT(3, 2, 6, 6)"| Transpose_1
    Transpose_1 -->|"FLOAT(3, 6, 2, 6)"| Relu_2

    O_Z(["Z FLOAT(a, b, c, d)"])
    Relu_2 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class MatMul_0,Transpose_1,Relu_2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(3, 2, 5, 6)"])
    I_X(["X FLOAT(3, 2, 6, 5)"])

    MatMul_0[["MatMul(., .)"]]
    Relu_1[["Relu(.)"]]
    Transpose_2[["Transpose(., perm=[0, 2, 1, 3])"]]

    I_X -->|"FLOAT(3, 2, 6, 5)"| MatMul_0
    I_Y -->|"FLOAT(3, 2, 5, 6)"| MatMul_0
    MatMul_0 -->|"FLOAT(3, 2, 6, 6)"| Relu_1
    Relu_1 -->|"FLOAT(3, 2, 6, 6)"| Transpose_2

    O_Z(["Z FLOAT(a, b, c, d)"])
    Transpose_2 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class MatMul_0,Relu_1,Transpose_2 opNode

apply(g: GraphBuilder, mm_node: NodeProto, tr_node: NodeProto, f_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult], left_first: bool = True) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.TransposeMatMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence Transpose, Matmul or Gemm into Gemm

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(128, 64)"])
    I_X(["X FLOAT(128, 32)"])

    Transpose_0[["Transpose(., perm=[1, 0])"]]
    MatMul_1[["MatMul(., .)"]]

    I_X -->|"FLOAT(128, 32)"| Transpose_0
    Transpose_0 -->|"FLOAT(32, 128)"| MatMul_1
    I_Y -->|"FLOAT(128, 64)"| MatMul_1

    O_Z(["Z FLOAT(32, 64)"])
    MatMul_1 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class Transpose_0,MatMul_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(128, 64)"])
    I_X(["X FLOAT(128, 32)"])

    Gemm_0[["Gemm(., .)"]]

    I_X -->|"FLOAT(128, 32)"| Gemm_0
    I_Y -->|"FLOAT(128, 64)"| Gemm_0

    O_Z(["Z FLOAT(32, 64)"])
    Gemm_0 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class Gemm_0 opNode

apply(g: GraphBuilder, node_before_left: NodeProto | None, node_before_right: NodeProto | None, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_matmul.TransposeReshapeMatMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence Transpose, Reshape, Matmul into Reshape, Transpose, Matmul if possible. Another optimizer will optimizes this sequence by using Gemm or better.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_yts(["yts FLOAT(2, 2, 7, 3)"])
    I_Y(["Y FLOAT(4, 3, 7)"])
    I_X(["X FLOAT(2, 2, 5, 7)"])

    MatMul_0[["MatMul(., .)"]]
    Reshape_1[["Reshape(., [2, 2, 7, 3])"]]
    Transpose_2[["Transpose(., perm=[0, 2, 1])"]]

    I_X -->|"FLOAT(2, 2, 5, 7)"| MatMul_0
    Reshape_1 -->|"FLOAT(2, 2, 7, 3)"| MatMul_0
    Transpose_2 -->|"FLOAT(4, 7, 3)"| Reshape_1
    I_Y -->|"FLOAT(4, 3, 7)"| Transpose_2

    O_yts(["yts FLOAT(2, 2, 7, 3)"])
    Reshape_1 --> O_yts
    O_Z(["Z FLOAT(2, 2, 5, 3)"])
    MatMul_0 --> O_Z

    class I_yts,I_Y,I_X,O_yts,O_Z ioNode
    class MatMul_0,Reshape_1,Transpose_2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_yts(["yts FLOAT(2, 2, 7, 3)"])
    I_Y(["Y FLOAT(4, 3, 7)"])
    I_X(["X FLOAT(2, 2, 5, 7)"])

    Reshape_0[["Reshape(., [2, 2, 3, 7])"]]
    Transpose_1[["Transpose(., perm=[0, 1, 3, 2])"]]
    MatMul_2[["MatMul(., .)"]]

    I_Y -->|"FLOAT(4, 3, 7)"| Reshape_0
    Reshape_0 -->|"FLOAT(2, 2, 3, 7)"| Transpose_1
    I_X -->|"FLOAT(2, 2, 5, 7)"| MatMul_2
    Transpose_1 -->|"FLOAT(2, 2, 7, 3)"| MatMul_2

    O_yts(["yts FLOAT(2, 2, 7, 3)"])
    Transpose_1 --> O_yts
    O_Z(["Z FLOAT(2, 2, 5, 3)"])
    MatMul_2 --> O_Z

    class I_yts,I_Y,I_X,O_yts,O_Z ioNode
    class Reshape_0,Transpose_1,MatMul_2 opNode

apply(g: GraphBuilder, node: NodeProto, node_left: NodeProto | None, node_left_tr: NodeProto | None, node_right: NodeProto | None, node_right_tr: NodeProto | None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult], left_first: bool = True) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

yobx.xoptim.patterns.onnx_matmul#

This Page