.xoptim.patterns¶

class experimental_experiment.xoptim.patterns.AlmostDoNothingPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)[source]¶

Checks that a Expand is really needed.

apply(g: GraphBuilder, node: NodeProto) → List[NodeProto][source]¶

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:: nodes – nodes returned by method match, there are then removed
Returns:: nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None[source]¶

Determines nodes around node which can be rewritten.

Parameters:

g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.
node – the matching must determine if some nodes around this one are part of set of nodes this pattern optmizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.
matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.
A function doing the rewriting (usually method apply of the pattern class).
An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

experimental_experiment.xoptim.patterns.get_default_patterns(verbose: int = 0) → List[PatternOptimization][source]¶

Returns a default list of optimization patterns. It is equal to the following list.

<<<

from experimental_experiment.xoptim.patterns_api import pattern_table_doc
from experimental_experiment.xoptim.patterns import get_default_patterns

print(pattern_table_doc(get_default_patterns(), as_rst=True))

>>>

	name	short_name	priority	doc
0	BatchNormalizationPattern	BatchNormalization	0	Checks that a BatchNormalization is really needed. Model with nodes to be fused: .. gdot
1	BatchNormalizationTrainingPattern	BatchNormalizationTraining	0	Checks that a BatchNormalization in training mode can be avoided.
2	CastLayerNormalizationCastPattern	CastLayerNormalizationCast	1	Checks that a Cast is really needed around LayerNormalization. Model with nodes to be fused: .. gdot
3	CastPattern	Cast	0	Checks that a Cast is really needed. Model with nodes to be fused: .. gdot
4	CastCastBinaryPattern	CastCastBinary	1	Moves two cast operators beyond a binary operator The cast must cast from a float type to another float type. Model with nodes to be fused: .. gdot
5	CastCastPattern	CastCast	0	Checks that two consecutive cast can be avoided. Model with nodes to be fused: .. gdot
6	CastOpCastPattern	CastOpCast	1	Removes two cast surrounding another operator. Model with nodes to be fused: .. gdot
7	ClipClipPattern	ClipClip	1	Merges consecutive clips if one is defining min and the other max. Model with nodes to be fused: .. gdot
8	ComputationCastOpCastPattern	ComputationCastOpCast	1	Changes the computation type to make it faster if one of the inputs was just casted before. Model with nodes to be fused: .. gdot
9	ConcatEmptyPattern	ConcatEmpty	1	Checks if one of the concatenated values is empty. Model with nodes to be fused: .. gdot
10	ConcatGatherPattern	ConcatGather	0	Checks if Gather(Concat) can be replaced by Identity. Model with nodes to be fused: .. gdot
11	ConcatReshapePattern	ConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused: .. gdot
12	ConcatTwiceUnaryPattern	ConcatTwiceUnary	1	Sin(Concat(x,x)) -> Concat(Sin(x), Sin(x)). Model with nodes to be fused: .. gdot
13	ConstantToInitializerPattern	ConstantToInitializer	1	Replaces a node Constant by an initializer and a node Identity. Model with nodes to be fused: .. gdot
14	ConvBiasNullPattern	ConvBiasNull	0	Checks that a Conv has a null bias. Model with nodes to be fused: .. gdot
15	DropoutPattern	Dropout	1	Checks that a Cast is really needed. Model with nodes to be fused: .. gdot
16	ExpandPattern	Expand	0	Checks that a Expand is really needed. Model with nodes to be fused: .. gdot
17	ExpandBroadcastPattern	ExpandBroadcast	1	Checks that a Expand is really needed before an element wise operator. The objective is to save one allocation and let the next operator do the expansion by broadcasting one input. Model with nodes to be fused: .. gdot
18	ExpandSwapPattern	ExpandSwap	1	Tries to move a node Expand forward in the graph. Expand + Exp can be changed into Exp + Expand. Then Exp applies on a tensor of a smaller or equal size. Model with nodes to be fused: .. gdot
19	GathersSplitPattern	GathersSplit	1	Merges multiple parallel gather into a split followed by unsqueeze. Model with nodes to be fused: .. gdot
20	GeluPattern	Gelu	0	Detects the decomposed version of Gelu with Tanh .. math
21	IdentityPattern	Identity	0	Replaces operator such as Div(X, 1), Mul(X, 1), Add(X, 0), Sub(X, 0), Transpose(X, [0, 1, 2, …]) by identity nodes. Model with nodes to be fused: .. gdot
22	LayerNormalizationPattern	LayerNormalization	1	Fuses nodes of a LayerNormalization. Model with nodes to be fused: .. gdot
23	LayerNormalizationScalePattern	LayerNormalizationScale	1	Fused LayerNormalization, scale, bias just after. Model with nodes to be fused: .. gdot
24	LeakyReluPattern	LeakyRelu	0	Detects the decomposed version of LeakyRelu. Model with nodes to be fused: .. gdot
25	MulMulMulScalarPattern	MulMulMulScalar	1	Replaces the sequence {Div \| Mul} and {Div \| Mul} + {Div \| Mul} with {Div \| Mul} Mul. Model with nodes to be fused: .. gdot
26	ReduceReshapePattern	ReduceReshape	1	Replaces the sequence Reduce* Reshape if reshape is only introduces to deal with a dimension kept because keepdims=1. Model with nodes to be fused: .. gdot
27	ReduceSumNormalizePattern	ReduceSumNormalize	1	Nodes equivalent to a reduction. Model with nodes to be fused: .. gdot
28	ReshapePattern	Reshape	0	Checks that a Reshape is really needed.
29	ReshapeMatMulReshapePattern	ReshapeMatMulReshape	1	Replaces the sequence Reshape, Matmul, Reshape by Matmul. Model with nodes to be fused: .. gdot
30	Reshape2Of3Pattern	Reshape2Of3	1	Replaces the reshapes around element-wise operators. It can be 3 or 2 out of 3. Model with nodes to be fused: .. gdot
31	ReshapeReshapeBinaryPattern	ReshapeReshapeBinary	1	Moves two reshape operators beyond a binary operator if it is possible. Model with nodes to be fused: .. gdot
32	MatMulAddPattern	MatMulAdd	3	Replaces the sequence MatMul, Add into Gemm. By default, no reshape is allowed this happens only it is two dimensions. Model with nodes to be fused: .. gdot
33	GemmTransposePattern	GemmTranspose	1	Replaces Gemm (., constant) by Gemm(., constant’, transB=1) Model with nodes to be fused: .. gdot
34	MatMulReshape2Of3Pattern	MatMulReshape2Of3	1	Replaces the reshapes around a matmul It can be 3 or 2 out of 3. It is similar to `experimental_experiment.xoptim.patterns.onnx_reshape.Reshape2Of3Pattern`. Model with nodes to be fused: .. gdot
35	MulMulMatMulPattern	MulMulMatMul	1	Replaces `MatMul(ac, bd)` where c and d are constant scalar by `MatMul(a,b) * (c,d)`. Model with nodes to be fused: .. gdot
36	ShapeBasedReshapeIsSqueezePattern	ShapeBasedReshapeIsSqueeze	0	Replaces a replaces by a squeeze or unsqueeze pattern if possible. It is only available for `opset >= 18`. Model with nodes to be fused: .. gdot
37	ShapeBasedStaticExpandPattern	ShapeBasedStaticExpand	0	Compares input and output shapes to tell if the expand can uses a constant as a second input. Model with nodes to be fused: .. gdot
38	ShapeBasedConcatExpandPattern	ShapeBasedConcatExpand	1	Rewrites Expand(X, concat(…)) if possible. Model with nodes to be fused: .. gdot
39	ShapeBasedEditDistanceReshapePattern	ShapeBasedEditDistanceReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1 or 0. The pattern tries to align shape information to infer a static shape. Model with nodes to be fused: .. gdot
40	ShapeBasedIdentityPattern	ShapeBasedIdentity	0	If a slice leads to the same shape and the step is 1 then it is identity. In some cases, just known the same is enough to replace them. Model with nodes to be fused: .. gdot
41	ShapeBasedExpandBroadcastPattern	ShapeBasedExpandBroadcast	1	Similar to `experimental_experiment.xoptim.patterns.onnx_expand.ExpandBroadcastPattern`, but it allows dynamic shapes as well. It does not look into the second argument of Expand, it just infers than an expand is not needed for a binary operator following just after.
42	ShapeBasedExpandBroadcastMatMulPattern	ShapeBasedExpandBroadcastMatMul	1	Similar to `experimental_experiment.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern`, but works only with MatMul. Model with nodes to be fused: .. gdot
43	ShapeBasedExpandCastWhereSwapPattern	ShapeBasedExpandCastWhereSwap	1	Rewrites Where(Cast(X), X, cond). Model with nodes to be fused: .. gdot
44	ShapeBasedExpandSwapPattern	ShapeBasedExpandSwap	1	Tries to move a node Expand forward in the graph for a binary operator. The code is similar to `experimental_experiment.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern` Model with nodes to be fused: .. gdot
45	ShapeBasedMatMulToMulPattern	ShapeBasedMatMulToMul	1	MatMul can be replaced by Mul with broadcast. It makes it easier to detect optimization pattern with Expand operators. Model with nodes to be fused: .. gdot
46	ShapedBasedReshapePattern	ShapedBasedReshape	0	Checks that a Reshape is really needed based on the input shape. Model with nodes to be fused: .. gdot
47	ShapeBasedSameChildrenPattern	ShapeBasedSameChildren	0	Checks there is no duplicated node doing the same than another one beside. `experimental_experiment.xoptim.patterns.onnx_any.SameChildrenPattern` checks it is exactly the same. This one assumes it is exactly the same in some cases such expand (X, sh1) = expand(X, sh2) if the output shapes are the same. Model with nodes to be fused: .. gdot
48	ShapeBasedShapeShapeAddPattern	ShapeBasedShapeShapeAdd	0	Tries to find another way to get a dimension obtained with the addition of two.
49	ReshapeReshapePattern	ReshapeReshape	0	Replaces the sequence Reshape, Reshape by Reshape. Model with nodes to be fused: .. gdot
50	RotaryEmbeddingPattern	RotaryEmbedding	1	Fuses nodes matching RotaryEmbedding(23). Model with nodes to be fused: .. gdot
51	SameChildrenPattern	SameChildren	0	Checks there is no duplicated node doing the same than another one beside. Model with nodes to be fused: .. gdot
52	SameChildrenFromInputPattern	SameChildrenFromInput	0	Checks there is no duplicated node doing the same than another one beside and taking a model input as input. Model with nodes to be fused: .. gdot
53	SequenceConstructAtPattern	SequenceConstructAt	1	Replaces the sequence `SequenceConstruct(x1, x2, ...)` followed by `SequenceAt(seq, 0)`, `SequenceAt(seq, 1)`, … Model with nodes to be fused: .. gdot
54	SliceSlicePattern	SliceSlice	1	Merges consecutive slices if axis are disjoints. Model with nodes to be fused: .. gdot
55	SlicesSplitPattern	SlicesSplit	1	Merges multiple parallel slices into a split. Model with nodes to be fused: .. gdot
56	SoftmaxCrossEntropyLossCastPattern	SoftmaxCrossEntropyLossCast	0	Detects one decomposed version of SoftmaxCrossEntropyLoss. Model with nodes to be fused: .. gdot
57	SplitConcatPattern	SplitConcat	1	Replaces Split + Concat into identity if this is equivalent. Model with nodes to be fused: .. gdot
58	SqueezeAddPattern	SqueezeAdd	0	Replaces the sequence Add(Squeeze, Squeeze) by Squeeze(Add). Model with nodes to be fused: .. gdot
59	SqueezeBinaryUnsqueezePattern	SqueezeBinaryUnsqueeze	0	Replaces the sequence Squeeze Binary Unsqueeze) by Binary. Model with nodes to be fused: .. gdot
60	SqueezeUnsqueezePattern	SqueezeUnsqueeze	0	Replaces the sequence Squeeze, Unsqueeze by Identity or the other ways around. Model with nodes to be fused: .. gdot
61	StaticConcatReshapePattern	StaticConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused: .. gdot
62	Sub1MulPattern	Sub1Mul	1	Replaces the sequence (1 - X) x Y by Y - X x Y to avoid the creation of a constant in the graph. x means element wise multiplication. Model with nodes to be fused: .. gdot
63	SwapExpandReshapePattern	SwapExpandReshape	0	Checks if Expand + Reshape can be swapped. Model with nodes to be fused: .. gdot
64	SwapUnaryPattern	SwapUnary	0	Tries to move computation nodes before any transpose or reshape. That works for unary operator or equivalent to that. Model with nodes to be fused: .. gdot
65	SwapUnsqueezeTransposePattern	SwapUnsqueezeTranspose	0	Swaps Unsqueeze and Transpose. Model with nodes to be fused: .. gdot
66	SwitchOrderBinaryPattern	SwitchOrderBinary	1	If it makes sense, switches the order of two multiplications or two addtions if the broadcasting reduces one operator to a an insignificant number. Model with nodes to be fused: .. gdot
67	SwitchReshapeActivationPattern	SwitchReshapeActivation	1	Swiches Gelu and Reshape after a Gemm or a MatMul. Gelu can also be Gelu, Exp, Elu, Relu, Tan, Tanh, Cos, Cosh, Sin, Sinh, Erf, LeakyRelu, PRelu, Selu, Softmax, Softplus. Reshape can also be Transpose. Model with nodes to be fused: .. gdot
68	TransposeEqualReshapePattern	TransposeEqualReshape	1	Replaces a Transpose by a Reshape when switched dimensions are all equal to 1 but one. Model with nodes to be fused: .. gdot
69	TransposeGatherPattern	TransposeGather	0	Removes one unnecessary transpose followed by Gather with only one index. Model with nodes to be fused: .. gdot
70	TransposeMatMulPattern	TransposeMatMul	1	Replaces the sequence Transpose, Matmul or Gemm into Gemm Model with nodes to be fused: .. gdot
71	TransposeReshapeMatMulPattern	TransposeReshapeMatMul	1	Replaces the sequence Transpose, Reshape, Matmul into Reshape, Transpose, Matmul if possible. Another optimizer will optimizes this sequence by using Gemm or better. Model with nodes to be fused: .. gdot
72	TransposeReshapeTransposePattern	TransposeReshapeTranspose	0	Swaps Reshape and Transpose in a sequence such as this one:
73	TransposeTransposePattern	TransposeTranspose	0	Removes two consecutive transpose if the second one put the tensor in origin shape. Model with nodes to be fused: .. gdot
74	UnsqueezeEqualPattern	UnsqueezeEqual	1	Replaces the sequence R -> Equal -> Unsqueeze, R -> Unsqueeze, into R -> Unsqueeze -> Equal. Model with nodes to be fused: .. gdot
75	UnsqueezeReshapePattern	UnsqueezeReshape	0	Unsqueeze + Reshape into Unsqueeze at a different place if possible.
76	UnsqueezeUnsqueezePattern	UnsqueezeUnsqueeze	0	Replaces the sequence Unsqueeze, Unsqueeze by Unsqueeze. Model with nodes to be fused: .. gdot
77	RotaryConcatPartPattern	RotaryConcatPart	1	Optimizes the following pattern .. plot
78	FunctionAttentionPattern	FunctionAttention	0	Merges Attention nodes into a local function. Model with nodes to be fused: .. gdot
79	FunctionCausalMaskPattern	FunctionCausalMask	1	Fuses nodes matching CausalMask into a local function. .. runpython
80	FunctionCausalMaskMulAddPattern	FunctionCausalMaskMulAdd	1	Fuses nodes matching CausalMask into a local function. .. runpython
81	FunctionCosSinCachePattern	FunctionCosSinCache	1	Fuses nodes to simplify the creation of cos/sin caches in LLM. .. runpython
82	FunctionHalfRotaryEmbeddingPattern	FunctionHalfRotaryEmbedding	1	Fuses nodes matching half RotaryEmbedding(23) into a local function. .. runpython
83	RMSNormalizationPattern	RMSNormalization	1	Fuses the nodes equivalent to RMSNormalization(23). Model with nodes to be fused: .. gdot