yobx.xoptim.patterns_ort.simplified_layer_normalization — yet-another-onnx-builder 0.1.0 documentation

class yobx.xoptim.patterns_ort.simplified_layer_normalization.SimplifiedLayerNormalizationMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence SimplifiedLayerNormalization + Mul by SimplifiedLayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_xs(["xs FLOAT(batch, cache, 192)"])
    I_weights(["weights FLOAT(192)"])

    Constant_0[["Constant() -#gt; scale"]]
    Constant_1[["Constant() -#gt; weights"]]
    skip_layer_norm2[["SimplifiedLayerNormalization(., ., axis=-1)"]]
    Mul_3[["Mul(., .)"]]

    I_xs -->|"FLOAT(batch, cache, 192)"| skip_layer_norm2
    Constant_0 -->|"FLOAT(192)"| skip_layer_norm2
    skip_layer_norm2 -->|"FLOAT(batch, cache, 192)"| Mul_3
    Constant_1 -->|"FLOAT(192)"| Mul_3

    O_a(["a FLOAT()"])
    Mul_3 --> O_a

    class I_xs,I_weights,O_a ioNode
    class Constant_0,Constant_1 constNode
    class skip_layer_norm2,Mul_3 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_xs(["xs FLOAT(batch, cache, 192)"])
    I_weights(["weights FLOAT(192)"])

    skip_layer_norm0[["SimplifiedLayerNormalization(., ., axis=-1)"]]

    I_xs -->|"FLOAT(batch, cache, 192)"| skip_layer_norm0
    I_weights -->|"FLOAT(192)"| skip_layer_norm0

    O_a(["a FLOAT()"])
    skip_layer_norm0 --> O_a

    class I_xs,I_weights,O_a ioNode
    class skip_layer_norm0 opNode

apply(g: GraphBuilder, simp_node: NodeProto, mul_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.simplified_layer_normalization.SimplifiedLayerNormalizationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Fuses the nodes equivalent to SimplifiedLayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(a, D)"])
    I_axis(["axis INT64(1)"])

    Constant_0[["Constant() -#gt; axis"]]
    Pow_1[["Pow(., [2.0])"]]
    ReduceMean_2[["ReduceMean(., .)"]]
    Add_3[["Add(., [1e-06])"]]
    Sqrt_4[["Sqrt(.)"]]
    Div_5[["Div([1.0], .)"]]
    Mul_6[["Mul(., .)"]]

    I_X -->|"FLOAT(a, D)"| Pow_1
    Pow_1 -->|"FLOAT(a, D)"| ReduceMean_2
    Constant_0 -->|"INT64(1)"| ReduceMean_2
    ReduceMean_2 -->|"FLOAT(a, 1)"| Add_3
    Add_3 -->|"FLOAT(a, 1)"| Sqrt_4
    Sqrt_4 -->|"FLOAT(a, 1)"| Div_5
    Div_5 -->|"FLOAT(a, 1)"| Mul_6
    I_X -->|"FLOAT(a, D)"| Mul_6

    O_Z(["Z FLOAT(a, 1)"])
    Div_5 --> O_Z
    O_Y(["Y FLOAT(a, D)"])
    Mul_6 --> O_Y

    class I_X,I_axis,O_Z,O_Y ioNode
    class Constant_0 constNode
    class Pow_1,ReduceMean_2,Add_3,Sqrt_4,Div_5,Mul_6 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(a, D)"])
    I_axis(["axis INT64(1)"])

    Shape_0[["Shape(.)"]]
    Gather_1[["Gather(., .)"]]
    ConstantOfShape_2[["ConstantOfShape(.)"]]
    skip_layer_norm3[["SimplifiedLayerNormalization(., ., axis=-1, stash_type=1)"]]

    I_X -->|"FLOAT(a, D)"| Shape_0
    Shape_0 -->|"INT64(2)"| Gather_1
    I_axis -->|"INT64(1)"| Gather_1
    Gather_1 -->|"INT64(1)"| ConstantOfShape_2
    I_X -->|"FLOAT(a, D)"| skip_layer_norm3
    ConstantOfShape_2 --> skip_layer_norm3

    O_Z(["Z FLOAT(a, 1)"])
    skip_layer_norm3 --> O_Z
    O_Y(["Y FLOAT(a, D)"])
    skip_layer_norm3 --> O_Y

    class I_X,I_axis,O_Z,O_Y ioNode
    class Shape_0,Gather_1,ConstantOfShape_2,skip_layer_norm3 opNode

apply(g: GraphBuilder, node_pow: NodeProto, node_reduce: NodeProto, node_add: NodeProto, node_sqrt: NodeProto, node_reciprocal: NodeProto, node_mul: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.simplified_layer_normalization.SkipLayerNormalizationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence Add + LayerNormalization into SkipLayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X2(["X2 FLOAT16(a, b, c)"])
    I_X1(["X1 FLOAT16(a, b, c)"])
    I_scale(["scale FLOAT16(c)"])
    I_bias(["bias FLOAT16(c)"])

    Add_0[["Add(., .)"]]
    LayerNormalization_1[["LayerNormalization(., ., ., axis=-1)"]]

    I_X1 -->|"FLOAT16(a, b, c)"| Add_0
    I_X2 -->|"FLOAT16(a, b, c)"| Add_0
    Add_0 -->|"FLOAT16(a, b, c)"| LayerNormalization_1
    I_scale -->|"FLOAT16(c)"| LayerNormalization_1
    I_bias -->|"FLOAT16(c)"| LayerNormalization_1

    O_add(["add FLOAT16(a, b, c)"])
    Add_0 --> O_add
    O_Y(["Y FLOAT16(a, b, c)"])
    LayerNormalization_1 --> O_Y

    class I_X2,I_X1,I_scale,I_bias,O_add,O_Y ioNode
    class Add_0,LayerNormalization_1 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X2(["X2 FLOAT16(a, b, c)"])
    I_X1(["X1 FLOAT16(a, b, c)"])
    I_scale(["scale FLOAT16(c)"])
    I_bias(["bias FLOAT16(c)"])

    SkipLayerNormalization_0[["com.microsoft.SkipLayerNormalization(., ., ., .)"]]

    I_X1 -->|"FLOAT16(a, b, c)"| SkipLayerNormalization_0
    I_X2 -->|"FLOAT16(a, b, c)"| SkipLayerNormalization_0
    I_scale -->|"FLOAT16(c)"| SkipLayerNormalization_0
    I_bias -->|"FLOAT16(c)"| SkipLayerNormalization_0

    O_add(["add FLOAT16(a, b, c)"])
    SkipLayerNormalization_0 --> O_add
    O_Y(["Y FLOAT16(a, b, c)"])
    SkipLayerNormalization_0 --> O_Y

    class I_X2,I_X1,I_scale,I_bias,O_add,O_Y ioNode
    class SkipLayerNormalization_0 opNode

apply(g: GraphBuilder, add_node: NodeProto, norm_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.simplified_layer_normalization.SkipSimplifiedLayerNormalizationMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence SkipSimplifiedLayerNormalization + Mul by SkipSimplifiedLayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(batch, cache, 192)"])
    I_skip(["skip FLOAT(batch, cache, 192)"])
    I_weights(["weights FLOAT(192)"])

    Constant_0[["Constant() -#gt; scale"]]
    Constant_1[["Constant() -#gt; weights"]]
    skip_layer_norm[["com.microsoft.SkipSimplifiedLayerNormalization(., ., .)"]]
    Mul_3[["Mul(., .)"]]

    I_X -->|"FLOAT(batch, cache, 192)"| skip_layer_norm
    I_skip -->|"FLOAT(batch, cache, 192)"| skip_layer_norm
    Constant_0 -->|"FLOAT(192)"| skip_layer_norm
    skip_layer_norm -->|"FLOAT(batch, cache, 192)"| Mul_3
    Constant_1 -->|"FLOAT(192)"| Mul_3

    O_a(["a FLOAT(batch, cache, 192)"])
    Mul_3 --> O_a
    O_xs(["xs FLOAT(batch, cache, 192)"])
    skip_layer_norm --> O_xs

    class I_X,I_skip,I_weights,O_a,O_xs ioNode
    class Constant_0,Constant_1 constNode
    class skip_layer_norm,Mul_3 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_X(["X FLOAT(batch, cache, 192)"])
    I_skip(["skip FLOAT(batch, cache, 192)"])
    I_weights(["weights FLOAT(192)"])

    layer_norm[["com.microsoft.SkipSimplifiedLayerNormalization(., ., .)"]]

    I_X -->|"FLOAT(batch, cache, 192)"| layer_norm
    I_skip -->|"FLOAT(batch, cache, 192)"| layer_norm
    I_weights -->|"FLOAT(192)"| layer_norm

    O_a(["a FLOAT(batch, cache, 192)"])
    layer_norm --> O_a
    O_xs(["xs FLOAT(batch, cache, 192)"])
    layer_norm --> O_xs

    class I_X,I_skip,I_weights,O_a,O_xs ioNode
    class layer_norm opNode

apply(g: GraphBuilder, skip_simp_node: NodeProto, mul_node: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class yobx.xoptim.patterns_ort.simplified_layer_normalization.SkipSimplifiedLayerNormalizationPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]#

Replaces the sequence Add + SimplifiedLayerNormalization by SkipSimplifiedLayerNormalization.

Model with nodes to be fused:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(192)"])
    I_skip(["skip FLOAT(batch, cache, 192)"])
    I_X(["X FLOAT(batch, cache, 192)"])

    Constant_0[["Constant() -#gt; scale"]]
    Add_1[["Add(., .)"]]
    skip_layer_norm2[["SimplifiedLayerNormalization(., ., axis=-1)"]]

    I_X -->|"FLOAT(batch, cache, 192)"| Add_1
    I_skip -->|"FLOAT(batch, cache, 192)"| Add_1
    Add_1 -->|"FLOAT(batch, cache, 192)"| skip_layer_norm2
    Constant_0 -->|"FLOAT(192)"| skip_layer_norm2

    O_xs(["xs FLOAT(batch, cache, 192)"])
    Add_1 --> O_xs
    O_ym(["ym FLOAT(batch, cache, 192)"])
    skip_layer_norm2 --> O_ym

    class I_scale,I_skip,I_X,O_xs,O_ym ioNode
    class Constant_0 constNode
    class Add_1,skip_layer_norm2 opNode

Outcome of the fusion:

        graph TD

    classDef ioNode fill:#dfd,stroke:#333,color:#333
    classDef initNode fill:#cccc00,stroke:#333,color:#333
    classDef constNode fill:#f9f,stroke:#333,stroke-width:2px,color:#333
    classDef opNode fill:#bbf,stroke:#333,stroke-width:2px,color:#333

    I_scale(["scale FLOAT(192)"])
    I_skip(["skip FLOAT(batch, cache, 192)"])
    I_X(["X FLOAT(batch, cache, 192)"])

    skip_layer_norm[["com.microsoft.SkipSimplifiedLayerNormalization(., ., .)"]]

    I_X -->|"FLOAT(batch, cache, 192)"| skip_layer_norm
    I_skip -->|"FLOAT(batch, cache, 192)"| skip_layer_norm
    I_scale -->|"FLOAT(192)"| skip_layer_norm

    O_xs(["xs FLOAT(batch, cache, 192)"])
    skip_layer_norm --> O_xs
    O_ym(["ym FLOAT(batch, cache, 192)"])
    skip_layer_norm --> O_ym

    class I_scale,I_skip,I_X,O_xs,O_ym ioNode
    class skip_layer_norm opNode

apply(g: GraphBuilder, node_add: NodeProto, node_simplified: NodeProto) → List[NodeProto][source]#

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]#

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optimizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

yobx.xoptim.patterns_ort.simplified_layer_normalization#

This Page