yobx.xoptim.patterns.onnx_layer_normalization — yet-another-onnx-builder 0.1.0 documentation

class yobx.xoptim.patterns.onnx_layer_normalization.BatchNormalizationPattern(verbose: int = 0, priority: int = 0)[source]#

Checks that a BatchNormalization is really needed.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(1024, 16)"])

    Constant_0[["Constant() -#gt; scale"]]
    Constant_1[["Constant() -#gt; B"]]
    Constant_2[["Constant() -#gt; input_mean"]]
    Constant_3[["Constant() -#gt; input_var"]]
    BatchNormalization_4[["BatchNormalization(., ., ., ., .)"]]

    I_X -->|"FLOAT(1024, 16)"| BatchNormalization_4
    Constant_0 -->|"FLOAT(16)"| BatchNormalization_4
    Constant_1 -->|"FLOAT(16)"| BatchNormalization_4
    Constant_2 -->|"FLOAT(16)"| BatchNormalization_4
    Constant_3 -->|"FLOAT(16)"| BatchNormalization_4

    O_Y(["Y FLOAT(1024, 16)"])
    BatchNormalization_4 --> O_Y

    class I_X,O_Y ioNode
    class Constant_0,Constant_1,Constant_2,Constant_3 constNode
    class BatchNormalization_4 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(1024, 16)"])

    Identity_0[["Identity(.)"]]

    I_X -->|"FLOAT(1024, 16)"| Identity_0

    O_Y(["Y FLOAT(1024, 16)"])
    Identity_0 --> O_Y

    class I_X,O_Y ioNode
    class Identity_0 opNode

apply(g: GraphBuilder, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.BatchNormalizationTrainingPattern(verbose: int = 0, priority: int = 0)[source]#

Checks that a BatchNormalization in training mode can be avoided.

apply(g: GraphBuilder, node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.CastLayerNormalizationCastPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Checks that a Cast is really needed around GroupNormalization, LayerNormalization, RMSNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(3)"])
    I_bias(["bias FLOAT(3)"])
    I_X(["X FLOAT16(3, 3)"])

    Constant_0[["Constant() -#gt; scale"]]
    Constant_1[["Constant() -#gt; bias"]]
    Cast_2[["Cast(., to=FLOAT)"]]
    LayerNormalization_3[["LayerNormalization(., ., ., stash_type=1)"]]
    Cast_4[["Cast(., to=FLOAT16)"]]

    I_X -->|"FLOAT16(3, 3)"| Cast_2
    Cast_2 -->|"FLOAT(3, 3)"| LayerNormalization_3
    Constant_0 -->|"FLOAT(3)"| LayerNormalization_3
    Constant_1 -->|"FLOAT(3)"| LayerNormalization_3
    LayerNormalization_3 -->|"FLOAT(3, 3)"| Cast_4

    O_Y(["Y FLOAT16(3, 3)"])
    Cast_4 --> O_Y

    class I_scale,I_bias,I_X,O_Y ioNode
    class Constant_0,Constant_1 constNode
    class Cast_2,LayerNormalization_3,Cast_4 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(3)"])
    I_bias(["bias FLOAT(3)"])
    I_X(["X FLOAT16(3, 3)"])

    Cast_0[["Cast(., to=FLOAT16)"]]
    Cast_1[["Cast(., to=FLOAT16)"]]
    LayerNormalization_2[["LayerNormalization(., ., ., stash_type=1)"]]

    I_scale -->|"FLOAT(3)"| Cast_0
    I_bias -->|"FLOAT(3)"| Cast_1
    I_X -->|"FLOAT16(3, 3)"| LayerNormalization_2
    Cast_0 -->|"FLOAT16(3)"| LayerNormalization_2
    Cast_1 -->|"FLOAT16(3)"| LayerNormalization_2

    O_Y(["Y FLOAT16(3, 3)"])
    LayerNormalization_2 --> O_Y

    class I_scale,I_bias,I_X,O_Y ioNode
    class Cast_0,Cast_1,LayerNormalization_2 opNode

apply(g: GraphBuilder, cast_before: NodeProto, node: NodeProto, cast_after: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.LayerNormalizationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Fuses nodes of a LayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_add_1(["add_1 FLOAT16(4, 512, 128)"])

    ReduceMean_0[["ReduceMean(., [-1])"]]
    Sub_1[["Sub(., .)"]]
    Pow_2[["Pow(., [2.0])"]]
    ReduceMean_3[["ReduceMean(., [-1])"]]
    Add_4[["Add(., [0.0])"]]
    Sqrt_5[["Sqrt(.)"]]
    Div_6[["Div(., .)"]]

    I_add_1 -->|"FLOAT16(4, 512, 128)"| ReduceMean_0
    I_add_1 -->|"FLOAT16(4, 512, 128)"| Sub_1
    ReduceMean_0 -->|"FLOAT16(4, 512, 1)"| Sub_1
    Sub_1 -->|"FLOAT16(4, 512, 128)"| Pow_2
    Pow_2 -->|"FLOAT16(4, 512, 128)"| ReduceMean_3
    ReduceMean_3 -->|"FLOAT16(4, 512, 1)"| Add_4
    Add_4 -->|"FLOAT16(4, 512, 1)"| Sqrt_5
    Sub_1 -->|"FLOAT16(4, 512, 128)"| Div_6
    Sqrt_5 -->|"FLOAT16(4, 512, 1)"| Div_6

    O__onx_div0(["_onx_div0 FLOAT16(4, 512, 128)"])
    Div_6 --> O__onx_div0

    class I_add_1,O__onx_div0 ioNode
    class ReduceMean_0,Sub_1,Pow_2,ReduceMean_3,Add_4,Sqrt_5,Div_6 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_add_1(["add_1 FLOAT16(4, 512, 128)"])

    Constant_0[["Constant() -#gt; p_model_albert_embeddings_layernorm_weight"]]
    Constant_1[["Constant() -#gt; p_model_albert_embeddings_layernorm_bias"]]
    LayerNormalization_2[["LayerNormalization(., ., ., axis=-1, stash_type=1)"]]

    I_add_1 -->|"FLOAT16(4, 512, 128)"| LayerNormalization_2
    Constant_0 -->|"FLOAT16(128)"| LayerNormalization_2
    Constant_1 -->|"FLOAT16(128)"| LayerNormalization_2

    O__onx_div0(["_onx_div0 FLOAT16(4, 512, 128)"])
    LayerNormalization_2 --> O__onx_div0

    class I_add_1,O__onx_div0 ioNode
    class Constant_0,Constant_1 constNode
    class LayerNormalization_2 opNode

apply(g: GraphBuilder, red: NodeProto, sub: NodeProto, pow: NodeProto, node: NodeProto, add: NodeProto | None, sqrt: NodeProto, div: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.LayerNormalizationScalePattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Fused LayerNormalization, scale, bias just after.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(3)"])
    I_X(["X FLOAT(a, b)"])
    I_s0(["s0 FLOAT(3)"])

    Constant_0[["Constant() -#gt; s0"]]
    Constant_1[["Constant() -#gt; scale"]]
    LayerNormalization_2[["LayerNormalization(., .)"]]
    Mul_3[["Mul(., .)"]]

    I_X -->|"FLOAT(a, b)"| LayerNormalization_2
    Constant_0 -->|"FLOAT(3)"| LayerNormalization_2
    LayerNormalization_2 -->|"FLOAT(a, b)"| Mul_3
    Constant_1 -->|"FLOAT(3)"| Mul_3

    O_Y(["Y FLOAT(a, b)"])
    Mul_3 --> O_Y

    class I_scale,I_X,I_s0,O_Y ioNode
    class Constant_0,Constant_1 constNode
    class LayerNormalization_2,Mul_3 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(3)"])
    I_X(["X FLOAT(a, b)"])
    I_s0(["s0 FLOAT(3)"])

    Mul_0[["Mul(., .)"]]
    LayerNormalization_1[["LayerNormalization(., .)"]]

    I_s0 -->|"FLOAT(3)"| Mul_0
    I_scale -->|"FLOAT(3)"| Mul_0
    I_X -->|"FLOAT(a, b)"| LayerNormalization_1
    Mul_0 -->|"FLOAT(3)"| LayerNormalization_1

    O_Y(["Y FLOAT(a, b)"])
    LayerNormalization_1 --> O_Y

    class I_scale,I_X,I_s0,O_Y ioNode
    class Mul_0,LayerNormalization_1 opNode

apply(g: GraphBuilder, ln_node: NodeProto, mul_node: NodeProto, add_node: NodeProto | None) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.RMSNormalizationMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Fuses the nodes RMSNormalization(23) + Mul into RMSNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(a, 2)"])

    RMSNormalization_0[["RMSNormalization(., [3.0, 4.0])"]]
    Mul_1[["Mul(., [3.0, 4.0])"]]

    I_X -->|"FLOAT(a, 2)"| RMSNormalization_0
    RMSNormalization_0 -->|"FLOAT(a, 2)"| Mul_1

    O_Y(["Y FLOAT(a, 2)"])
    Mul_1 --> O_Y

    class I_X,O_Y ioNode
    class RMSNormalization_0,Mul_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(a, 2)"])

    RMSNormalization_0[["RMSNormalization(., [9.0, 16.0])"]]

    I_X -->|"FLOAT(a, 2)"| RMSNormalization_0

    O_Y(["Y FLOAT(a, 2)"])
    RMSNormalization_0 --> O_Y

    class I_X,O_Y ioNode
    class RMSNormalization_0 opNode

apply(g: GraphBuilder, rms_node: NodeProto, mul_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns.onnx_layer_normalization.RMSNormalizationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Fuses the nodes equivalent to RMSNormalization(23).

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT16(a, D)"])
    I_axis(["axis INT64(1)"])

    Constant_0[["Constant() -#gt; axis"]]
    Cast_1[["Cast(., to=FLOAT)"]]
    Pow_2[["Pow(., [2.0])"]]
    ReduceMean_3[["ReduceMean(., .)"]]
    Add_4[["Add(., [1e-06])"]]
    Sqrt_5[["Sqrt(.)"]]
    Div_6[["Div([1.0], .)"]]
    Mul_7[["Mul(., .)"]]
    Cast_8[["Cast(., to=FLOAT16)"]]

    I_X -->|"FLOAT16(a, D)"| Cast_1
    Cast_1 -->|"FLOAT(a, D)"| Pow_2
    Pow_2 -->|"FLOAT(a, D)"| ReduceMean_3
    Constant_0 -->|"INT64(1)"| ReduceMean_3
    ReduceMean_3 -->|"FLOAT(a, 1)"| Add_4
    Add_4 -->|"FLOAT(a, 1)"| Sqrt_5
    Sqrt_5 -->|"FLOAT(a, 1)"| Div_6
    Div_6 -->|"FLOAT(a, 1)"| Mul_7
    Cast_1 -->|"FLOAT(a, D)"| Mul_7
    Mul_7 -->|"FLOAT(a, D)"| Cast_8

    O_Y(["Y FLOAT16(a, D)"])
    Cast_8 --> O_Y

    class I_X,I_axis,O_Y ioNode
    class Constant_0 constNode
    class Cast_1,Pow_2,ReduceMean_3,Add_4,Sqrt_5,Div_6,Mul_7,Cast_8 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT16(a, D)"])
    I_axis(["axis INT64(1)"])

    Shape_0[["Shape(.)"]]
    Gather_1[["Gather(., .)"]]
    ConstantOfShape_2[["ConstantOfShape(.)"]]
    RMSNormalization_3[["RMSNormalization(., ., axis=-1, stash_type=1)"]]

    I_X -->|"FLOAT16(a, D)"| Shape_0
    Shape_0 -->|"INT64(2)"| Gather_1
    I_axis -->|"INT64(1)"| Gather_1
    Gather_1 -->|"INT64(1)"| ConstantOfShape_2
    I_X -->|"FLOAT16(a, D)"| RMSNormalization_3
    ConstantOfShape_2 --> RMSNormalization_3

    O_Y(["Y FLOAT16(a, D)"])
    RMSNormalization_3 --> O_Y

    class I_X,I_axis,O_Y ioNode
    class Shape_0,Gather_1,ConstantOfShape_2,RMSNormalization_3 opNode

apply(g: GraphBuilder, cast_1: NodeProto, node_pow: NodeProto, node_reduce: NodeProto, node_add: NodeProto, _node_sqrt: NodeProto, _node_reciprocal: NodeProto, node_mul: NodeProto, cast_2: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

yobx.xoptim.patterns.onnx_layer_normalization#

This Page