yobx.xoptim.patterns_ort.fused_matmul#

class yobx.xoptim.patterns_ort.fused_matmul.FusedMatMulDivPattern(verbose: int = 0, priority: int = 2)[source]#

Replaces the Matmul, Div into FusedMatMul.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(2, 2, 128, 64)"])
    I_X(["X FLOAT(2, 2, 32, 128)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]
    Div_1[["Div(., [2.0])"]]

    I_X -->|"FLOAT(2, 2, 32, 128)"| FusedMatMul_0
    I_Y -->|"FLOAT(2, 2, 128, 64)"| FusedMatMul_0
    FusedMatMul_0 -->|"FLOAT(2, 2, 32, 64)"| Div_1

    O_Z(["Z FLOAT(2, 2, 32, 64)"])
    Div_1 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class FusedMatMul_0,Div_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_Y(["Y FLOAT(2, 2, 128, 64)"])
    I_X(["X FLOAT(2, 2, 32, 128)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]

    I_X -->|"FLOAT(2, 2, 32, 128)"| FusedMatMul_0
    I_Y -->|"FLOAT(2, 2, 128, 64)"| FusedMatMul_0

    O_Z(["Z FLOAT(2, 2, 32, 64)"])
    FusedMatMul_0 --> O_Z

    class I_Y,I_X,O_Z ioNode
    class FusedMatMul_0 opNode
apply(g: GraphBuilder, node: NodeProto, node_div: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.FusedMatMulPattern(verbose: int = 0, priority: int = 2)[source]#

Replaces the sequence Transpose, Matmul into FusedMatMul.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 32, 128)"])
    I_Y(["Y FLOAT(2, 2, 64, 128)"])

    Transpose_0[["Transpose(., perm=[0, 1, 3, 2])"]]
    MatMul_1[["MatMul(., .)"]]

    I_X -->|"FLOAT(2, 2, 32, 128)"| Transpose_0
    I_Y -->|"FLOAT(2, 2, 64, 128)"| MatMul_1
    Transpose_0 -->|"FLOAT(2, 2, 128, 32)"| MatMul_1

    O_Z(["Z FLOAT(2, 2, 64, 32)"])
    MatMul_1 --> O_Z

    class I_X,I_Y,O_Z ioNode
    class Transpose_0,MatMul_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 32, 128)"])
    I_Y(["Y FLOAT(2, 2, 64, 128)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]

    I_Y -->|"FLOAT(2, 2, 64, 128)"| FusedMatMul_0
    I_X -->|"FLOAT(2, 2, 32, 128)"| FusedMatMul_0

    O_Z(["Z FLOAT(2, 2, 64, 32)"])
    FusedMatMul_0 --> O_Z

    class I_X,I_Y,O_Z ioNode
    class FusedMatMul_0 opNode
apply(g: GraphBuilder, node_before_left: NodeProto | None, node_before_right: NodeProto | None, node: NodeProto, scale: NodeProto | None = None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.FusedMatMulTransposePattern(verbose: int = 0, priority: int = 3)[source]#

Replaces the sequence (Fused)Matmul(A,B) + Transpose into FusedMatMul(B.T, A.T).

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 6, 3)"])
    I_Y(["Y FLOAT(2, 2, 5, 6)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]
    Transpose_1[["Transpose(., perm=[0, 1, 3, 2])"]]

    I_X -->|"FLOAT(2, 2, 6, 3)"| FusedMatMul_0
    I_Y -->|"FLOAT(2, 2, 5, 6)"| FusedMatMul_0
    FusedMatMul_0 -->|"FLOAT(2, 2, 3, 5)"| Transpose_1

    O_Z(["Z FLOAT(2, 2, UNKNOWNDIM, UNKNOWNDIM1)"])
    Transpose_1 --> O_Z

    class I_X,I_Y,O_Z ioNode
    class FusedMatMul_0,Transpose_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 6, 3)"])
    I_Y(["Y FLOAT(2, 2, 5, 6)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]

    I_Y -->|"FLOAT(2, 2, 5, 6)"| FusedMatMul_0
    I_X -->|"FLOAT(2, 2, 6, 3)"| FusedMatMul_0

    O_Z(["Z FLOAT(2, 2, UNKNOWNDIM, UNKNOWNDIM1)"])
    FusedMatMul_0 --> O_Z

    class I_X,I_Y,O_Z ioNode
    class FusedMatMul_0 opNode
apply(g: GraphBuilder, node: NodeProto, transpose_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.FusedMatMulx2Pattern(verbose: int = 0, priority: int = 3)[source]#

Replaces the sequence Div by a scalar consumed by two FusedMatMul.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 4, 4)"])

    Div_0[["Div(., [2.0])"]]
    FusedMatMul_1[["com.microsoft.FusedMatMul(., .)"]]
    FusedMatMul_2[["com.microsoft.FusedMatMul(., .)"]]

    I_X -->|"FLOAT(2, 2, 4, 4)"| Div_0
    Div_0 -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_1
    I_X -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_1
    I_X -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_2
    Div_0 -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_2

    O_x2(["x2 FLOAT(2, 2, 4, 4)"])
    FusedMatMul_2 --> O_x2
    O_x1(["x1 FLOAT(2, 2, 4, 4)"])
    FusedMatMul_1 --> O_x1

    class I_X,O_x2,O_x1 ioNode
    class Div_0,FusedMatMul_1,FusedMatMul_2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(2, 2, 4, 4)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]
    FusedMatMul_1[["com.microsoft.FusedMatMul(., .)"]]

    I_X -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_0
    I_X -->|"FLOAT(2, 2, 4, 4)"| FusedMatMul_1

    O_x2(["x2 FLOAT(2, 2, 4, 4)"])
    FusedMatMul_1 --> O_x2
    O_x1(["x1 FLOAT(2, 2, 4, 4)"])
    FusedMatMul_0 --> O_x1

    class I_X,O_x2,O_x1 ioNode
    class FusedMatMul_0,FusedMatMul_1 opNode
apply(g: GraphBuilder, div_node: NodeProto | None, *mnodes: NodeProto | None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.ReshapeGemmPattern(verbose: int = 0, priority: int = 3)[source]#

Replaces the sequence Reshape(-1, …) + Gemm into FusedMatMul().

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(4, 8)"])
    I_A(["A FLOAT(a, b, 8)"])

    Reshape_0[["Reshape(., [-1, 8])"]]
    Gemm_1[["Gemm(., .)"]]

    I_A -->|"FLOAT(a, b, 8)"| Reshape_0
    Reshape_0 -->|"FLOAT(a*b, 8)"| Gemm_1
    I_B -->|"FLOAT(4, 8)"| Gemm_1

    O_Y(["Y FLOAT(f, g)"])
    Gemm_1 --> O_Y

    class I_B,I_A,O_Y ioNode
    class Reshape_0,Gemm_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(4, 8)"])
    I_A(["A FLOAT(a, b, 8)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]
    Reshape_1[["Reshape(., [-1, 4])"]]

    I_A -->|"FLOAT(a, b, 8)"| FusedMatMul_0
    I_B -->|"FLOAT(4, 8)"| FusedMatMul_0
    FusedMatMul_0 -->|"FLOAT(a, b, 4)"| Reshape_1

    O_Y(["Y FLOAT(f, g)"])
    Reshape_1 --> O_Y

    class I_B,I_A,O_Y ioNode
    class FusedMatMul_0,Reshape_1 opNode
apply(g: GraphBuilder, reshape_node: NodeProto, gemm_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.ReshapeGemmReshapePattern(verbose: int = 0, priority: int = 3)[source]#

Replaces the sequence Reshape + Gemm + Reshape into FusedMatMul.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(8, 4)"])
    I_A(["A FLOAT(a, b, c)"])
    I_shapey(["shapey INT64(e)"])

    Reshape_0[["Reshape(., [-1, 8])"]]
    Gemm_1[["Gemm(., .)"]]
    Reshape_2[["Reshape(., .)"]]

    I_A -->|"FLOAT(a, b, c)"| Reshape_0
    Reshape_0 -->|"FLOAT(a*b*c//8, 8)"| Gemm_1
    I_B -->|"FLOAT(8, 4)"| Gemm_1
    Gemm_1 -->|"FLOAT(a*b*c//8, 4)"| Reshape_2
    I_shapey -->|"INT64(e)"| Reshape_2

    O_Y(["Y FLOAT(a, b, c)"])
    Reshape_2 --> O_Y

    class I_B,I_A,I_shapey,O_Y ioNode
    class Reshape_0,Gemm_1,Reshape_2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(8, 4)"])
    I_A(["A FLOAT(a, b, c)"])

    FusedMatMul_0[["com.microsoft.FusedMatMul(., .)"]]

    I_A -->|"FLOAT(a, b, c)"| FusedMatMul_0
    I_B -->|"FLOAT(8, 4)"| FusedMatMul_0

    O_Y(["Y FLOAT(a, b, c)"])
    FusedMatMul_0 --> O_Y

    class I_B,I_A,O_Y ioNode
    class FusedMatMul_0 opNode
apply(g: GraphBuilder, reshape_before: NodeProto, gemm_node: NodeProto, reshape_after: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.fused_matmul.TransposeFusedMatMulBPattern(verbose: int = 0, priority: int = 3)[source]#

Replaces the sequence Transpose(B, [0, 2, 3, 1] + (Fused)Matmul(A,B) into Transpose(A, [0, 2, 1, 3]) + FusedMatMul(A, B, transB=1).

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(i, j, k, l)"])
    I_A(["A FLOAT(a, b, c, d)"])

    Transpose_0[["Transpose(., perm=[0, 2, 3, 1])"]]
    MatMul_1[["MatMul(., .)"]]

    I_B -->|"FLOAT(i, j, k, l)"| Transpose_0
    I_A -->|"FLOAT(a, b, c, d)"| MatMul_1
    Transpose_0 -->|"FLOAT(i, k, l, j)"| MatMul_1

    O_Y(["Y FLOAT(m, n, o, p)"])
    MatMul_1 --> O_Y

    class I_B,I_A,O_Y ioNode
    class Transpose_0,MatMul_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_B(["B FLOAT(i, j, k, l)"])
    I_A(["A FLOAT(a, b, c, d)"])

    Transpose_0[["Transpose(., perm=[0, 2, 1, 3])"]]
    FusedMatMul_1[["com.microsoft.FusedMatMul(., .)"]]

    I_B -->|"FLOAT(i, j, k, l)"| Transpose_0
    I_A -->|"FLOAT(a, b, c, d)"| FusedMatMul_1
    Transpose_0 -->|"FLOAT(i, k, j, l)"| FusedMatMul_1

    O_Y(["Y FLOAT(m, n, o, p)"])
    FusedMatMul_1 --> O_Y

    class I_B,I_A,O_Y ioNode
    class Transpose_0,FusedMatMul_1 opNode
apply(g: GraphBuilder, transpose_node: NodeProto, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

  • g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

  • node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

  • matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

  • a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

  • A function doing the rewriting (usually method apply of the pattern class).

  • An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.