yobx.xoptim.patterns#

modules

get_default_patterns#

yobx.xoptim.patterns.get_default_patterns(verbose: int = 0) → List[PatternOptimization][source]#

Returns a default list of optimization patterns. It is equal to the following list.

<<<

from yobx.xoptim.patterns_api import pattern_table_doc
from yobx.xoptim.patterns import get_default_patterns

print(pattern_table_doc(get_default_patterns(), as_rst=True))

>>>

	name	short_name	priority	doc
0	BatchNormalizationPattern	BatchNormalization	0	Checks that a BatchNormalization is really needed. Model with nodes to be fused…
1	BatchNormalizationTrainingPattern	BatchNormalizationTraining	0	Checks that a BatchNormalization in training mode can be avoided.
2	CastLayerNormalizationCastPattern	CastLayerNormalizationCast	1	Checks that a Cast is really needed around GroupNormalization, LayerNormalization, RMSNormalization. Model with nodes to be fused…
3	CastPattern	Cast	0	Checks that a Cast is really needed. Model with nodes to be fused…
4	CastCastBinaryPattern	CastCastBinary	1	Moves two cast operators beyond a binary operator The cast must cast from a float type to another float type. Model with nodes to be fused…
5	CastCastPattern	CastCast	0	Checks that two consecutive cast can be avoided. Model with nodes to be fused…
6	CastOpCastPattern	CastOpCast	1	Removes two cast surrounding another operator. Model with nodes to be fused…
7	ClipClipPattern	ClipClip	1	Merges consecutive clips if one is defining min and the other max. Model with nodes to be fused…
8	ConcatEmptyPattern	ConcatEmpty	1	Checks if one of the concatenated values is empty. Model with nodes to be fused…
9	ConcatGatherPattern	ConcatGather	0	Checks if Gather(Concat) can be replaced by Identity. Model with nodes to be fused…
10	ConcatReshapePattern	ConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused…
11	ConcatTwiceUnaryPattern	ConcatTwiceUnary	1	Sin(Concat(x,x)) -> Concat(Sin(x), Sin(x)). Model with nodes to be fused…
12	ConstantToInitializerPattern	ConstantToInitializer	1	Replaces a node Constant by an initializer and a node Identity. Model with nodes to be fused…
13	ConvBiasNullPattern	ConvBiasNull	0	Checks that a Conv has a null bias. Model with nodes to be fused…
14	DropoutPattern	Dropout	1	Checks that a Cast is really needed. Model with nodes to be fused…
15	ExpandPattern	Expand	0	Checks that a Expand is really needed. Model with nodes to be fused…
16	ExpandBroadcastPattern	ExpandBroadcast	1	Checks that a Expand is really needed before an element wise operator. The objective is to save one allocation and let the next operator do the expansion by broadcasting one input. Model with nodes to be fused…
17	ExpandSwapPattern	ExpandSwap	1	Tries to move a node Expand forward in the graph. Expand + Exp can be changed into Exp + Expand. Then Exp applies on a tensor of a smaller or equal size. Model with nodes to be fused…
18	ExpandUnsqueezeExpandPattern	ExpandUnsqueezeExpand	0	Fuses the sequence Expand + Unsqueeze + Expand into Unsqueeze + Expand. Since Expand does not change the rank of a tensor, the Unsqueeze axes are valid for the original tensor as well, and the final Expand can handle both the broadcasting of the first Expand and the new dimension added by Unsqueeze. Model with nodes to be fused…
19	GathersSplitPattern	GathersSplit	1	Merges multiple parallel gather into a split followed by unsqueeze. Model with nodes to be fused…
20	GeluPattern	Gelu	0	Detects the decomposed version of Gelu with Tanh
21	IdentityPattern	Identity	0	Replaces operator such as Div(X, 1), Mul(X, 1), Add(X, 0), Sub(X, 0), Transpose(X, [0, 1, 2, …]) by identity nodes. It looks into patterns involving the following operators…
22	LayerNormalizationPattern	LayerNormalization	1	Fuses nodes of a LayerNormalization. Model with nodes to be fused…
23	LayerNormalizationScalePattern	LayerNormalizationScale	1	Fused LayerNormalization, scale, bias just after. Model with nodes to be fused…
24	LeakyReluPattern	LeakyRelu	0	Detects the decomposed version of LeakyRelu. Model with nodes to be fused…
25	MaxReluPattern	MaxRelu	1	Replaces `Max(x, 0)` or `Max(0, x)` with `Relu(x)`. Model with nodes to be fused…
26	MulMulMulScalarPattern	MulMulMulScalar	1	Replaces the sequence {Div \| Mul} and {Div \| Mul} + {Div \| Mul} with {Div \| Mul} Mul. Model with nodes to be fused…
27	MulUnsqueezeUnsqueezePattern	MulUnsqueezeUnsqueeze	0	Replaces `Mul(Unsqueeze(x, axes), Unsqueeze(y, axes))` by `Unsqueeze(Mul(x, y), axes)` when both inputs are unsqueezed with the same axes. Model with nodes to be fused…
28	NotNotPattern	NotNot	1	Fuses Not + Not into Identity. Model with nodes to be fused…
29	NotWherePattern	NotWhere	1	Replaces the sequence Where(Not(cond), X, Y) -> Where(cond, Y, X).
30	ReduceArgTopKPattern	ReduceArgTopK	1	Fuses ReduceMin(X, axis), ArgMin(X, axis) into TopK(, k=1). Model with nodes to be fused…
31	ReduceReshapePattern	ReduceReshape	1	Replaces the sequence Reduce* Reshape if reshape is only introduces to deal with a dimension kept because keepdims=1. Model with nodes to be fused…
32	ReduceSumNormalizePattern	ReduceSumNormalize	1	Nodes equivalent to a reduction. Model with nodes to be fused…
33	ReshapePattern	Reshape	0	Checks that a Reshape is really needed.
34	ReshapeMatMulReshapePattern	ReshapeMatMulReshape	1	Replaces the sequence Reshape, Matmul, Reshape by Matmul. Model with nodes to be fused…
35	Reshape2Of3Pattern	Reshape2Of3	1	Replaces the reshapes around element-wise operators. It can be 3 or 2 out of 3. Model with nodes to be fused…
36	ReshapeReshapeBinaryPattern	ReshapeReshapeBinary	1	Moves two reshape operators beyond a binary operator if it is possible. Model with nodes to be fused…
37	MatMulAddPattern	MatMulAdd	3	Replaces the sequence MatMul, Add into Gemm. By default, no reshape is allowed this happens only it is two dimensions. Model with nodes to be fused…
38	GemmTransposePattern	GemmTranspose	1	Replaces Gemm (., constant) by Gemm(., constant’, transB=1) Model with nodes to be fused…
39	MatMulReshape2Of3Pattern	MatMulReshape2Of3	1	Replaces the reshapes around a matmul It can be 3 or 2 out of 3. It is similar to `yobx.xoptim.patterns.onnx_reshape.Reshape2Of3Pattern`. Model with nodes to be fused…
40	MulMulMatMulPattern	MulMulMatMul	1	Replaces `MatMul(ac, bd)` where c and d are constant scalar by `MatMul(a,b) * (c,d)`. Model with nodes to be fused…
41	ShapeBasedReshapeIsSqueezePattern	ShapeBasedReshapeIsSqueeze	0	Replaces a replaces by a squeeze or unsqueeze pattern if possible. It is only available for `opset >= 18`. Model with nodes to be fused…
42	ShapeBasedStaticExpandPattern	ShapeBasedStaticExpand	0	Compares input and output shapes to tell if the expand can uses a constant as a second input. Model with nodes to be fused…
43	ShapeBasedConcatExpandPattern	ShapeBasedConcatExpand	1	Rewrites Expand(X, concat(…)) if possible. Model with nodes to be fused…
44	ShapeBasedEditDistanceReshapePattern	ShapeBasedEditDistanceReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1 or 0. The pattern tries to align shape information to infer a static shape. Model with nodes to be fused…
45	ShapeBasedIdentityPattern	ShapeBasedIdentity	0	If a slice leads to the same shape and the step is 1 then it is identity. In some cases, just known the same is enough to replace them. Model with nodes to be fused…
46	ShapeBasedExpandBroadcastPattern	ShapeBasedExpandBroadcast	1	Similar to `yobx.xoptim.patterns.onnx_expand.ExpandBroadcastPattern`, but it allows dynamic shapes as well. It does not look into the second argument of Expand, it just infers than an expand is not needed for a binary operator following just after. Model with nodes to be fused…
47	ShapeBasedExpandBroadcastMatMulPattern	ShapeBasedExpandBroadcastMatMul	1	Similar to `yobx.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern`, but works only with MatMul. Model with nodes to be fused…
48	ShapeBasedExpandCastWhereSwapPattern	ShapeBasedExpandCastWhereSwap	1	Rewrites Where(Cast(X), X, cond). Model with nodes to be fused…
49	ShapeBasedExpandSwapPattern	ShapeBasedExpandSwap	1	Tries to move a node Expand forward in the graph for a binary operator. The code is similar to `yobx.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern` Model with nodes to be fused…
50	ShapeBasedMatMulToMulPattern	ShapeBasedMatMulToMul	1	MatMul can be replaced by Mul with broadcast. It makes it easier to detect optimization pattern with Expand operators. Model with nodes to be fused…
51	ShapedBasedReshapePattern	ShapedBasedReshape	0	Checks that a Reshape is really needed based on the input shape. Model with nodes to be fused…
52	ShapeBasedSameChildrenPattern	ShapeBasedSameChildren	0	Checks there is no duplicated node doing the same than another one beside. `yobx.xoptim.patterns.onnx_any.SameChildrenPattern` checks it is exactly the same. This one assumes it is exactly the same in some cases such expand (X, sh1) = expand(X, sh2) if the output shapes are the same. Model with nodes to be fused…
53	ShapeBasedShapeShapeAddPattern	ShapeBasedShapeShapeAdd	0	Tries to find another way to get a dimension obtained with the addition of two.
54	ReshapeReshapePattern	ReshapeReshape	0	Replaces the sequence Reshape, Reshape by Reshape. Model with nodes to be fused…
55	RotaryEmbeddingPattern	RotaryEmbedding	1	Fuses nodes matching RotaryEmbedding(23). Model with nodes to be fused…
56	SameChildrenPattern	SameChildren	0	Checks there is no duplicated node doing the same than another one beside. Model with nodes to be fused…
57	SameChildrenFromInputPattern	SameChildrenFromInput	0	Checks there is no duplicated node doing the same than another one beside and taking a model input as input. Model with nodes to be fused…
58	SequenceConstructAtPattern	SequenceConstructAt	1	Replaces the sequence `SequenceConstruct(x1, x2, ...)` followed by `SequenceAt(seq, 0)`, `SequenceAt(seq, 1)`, … Model with nodes to be fused…
59	SliceSlicePattern	SliceSlice	1	Merges consecutive slices if axis are disjoints. Model with nodes to be fused…
60	SlicesSplitPattern	SlicesSplit	1	Merges multiple parallel slices into a split. Model with nodes to be fused…
61	SoftmaxCrossEntropyLossCastPattern	SoftmaxCrossEntropyLossCast	0	Detects one decomposed version of SoftmaxCrossEntropyLoss. Model with nodes to be fused…
62	SplitConcatPattern	SplitConcat	1	Replaces Split + Concat into identity if this is equivalent. Model with nodes to be fused…
63	SqueezeAddPattern	SqueezeAdd	0	Replaces the sequence Add(Squeeze, Squeeze) by Squeeze(Add). Model with nodes to be fused…
64	SqueezeBinaryUnsqueezePattern	SqueezeBinaryUnsqueeze	0	Replaces the sequence Squeeze Binary Unsqueeze) by Binary. Model with nodes to be fused…
65	SqueezeUnsqueezePattern	SqueezeUnsqueeze	0	Replaces the sequence Squeeze, Unsqueeze by Identity or the other ways around. Model with nodes to be fused…
66	StaticConcatReshapePattern	StaticConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused…
67	Sub1MulPattern	Sub1Mul	1	Replaces the sequence (1 - X) x Y by Y - X x Y to avoid the creation of a constant in the graph. x means element wise multiplication. Model with nodes to be fused…
68	SwapExpandReshapePattern	SwapExpandReshape	0	Checks if Expand + Reshape can be swapped. Model with nodes to be fused…
69	SwapExpandUnsqueezePattern	SwapExpandUnsqueeze	0	Swaps Expand and Unsqueeze when Unsqueeze directly follows Expand. `Expand(X, shape) → Unsqueeze(expanded, axes)` is rewritten as `Unsqueeze(X, axes) → Expand(unsqueezed, new_shape)` where `new_shape` is obtained by inserting `1` at every position listed in `axes` into the original expand shape. Performing the Unsqueeze before the Expand means the Unsqueeze operates on the smaller (pre-expanded) tensor, which is more efficient. Model with nodes to be fused…
70	SwapRangeAddScalarPattern	SwapRangeAddScalar	1	Swap Range + Add when a scalar is added. Model with nodes to be fused…
71	SwapUnaryPattern	SwapUnary	0	Tries to move computation nodes before any transpose or reshape. That works for unary operator or equivalent to that. Model with nodes to be fused…
72	SwapUnsqueezeTransposePattern	SwapUnsqueezeTranspose	0	Swaps Unsqueeze and Transpose. Model with nodes to be fused…
73	SwitchOrderBinaryPattern	SwitchOrderBinary	1	If it makes sense, switches the order of two multiplications or two additions if the broadcasting reduces one operator to a an insignificant number. Model with nodes to be fused…
74	SwitchReshapeActivationPattern	SwitchReshapeActivation	1	Switches Gelu and Reshape after a Gemm or a MatMul. Gelu can also be Gelu, Exp, Elu, Relu, Tan, Tanh, Cos, Cosh, Sin, Sinh, Erf, LeakyRelu, PRelu, Selu, Softmax, Softplus. Reshape can also be Transpose. Model with nodes to be fused…
75	TransposeEqualReshapePattern	TransposeEqualReshape	1	Replaces a Transpose by a Reshape when switched dimensions are all equal to 1 but one. Model with nodes to be fused…
76	TransposeGatherPattern	TransposeGather	0	Removes one unnecessary transpose followed by Gather with only one index. Model with nodes to be fused…
77	TransposeMatMulPattern	TransposeMatMul	1	Replaces the sequence Transpose, Matmul or Gemm into Gemm Model with nodes to be fused…
78	TransposeReshapeMatMulPattern	TransposeReshapeMatMul	1	Replaces the sequence Transpose, Reshape, Matmul into Reshape, Transpose, Matmul if possible. Another optimizer will optimizes this sequence by using Gemm or better. Model with nodes to be fused…
79	TransposeReshapeTransposePattern	TransposeReshapeTranspose	0	Swaps Reshape and Transpose in a sequence such as this one…
80	TransposeTransposePattern	TransposeTranspose	0	Removes two consecutive transpose if the second one put the tensor in origin shape. Model with nodes to be fused…
81	UnsqueezeEqualPattern	UnsqueezeEqual	1	Replaces the sequence R -> Equal -> Unsqueeze, R -> Unsqueeze, into R -> Unsqueeze -> Equal. Model with nodes to be fused…
82	UnsqueezeOrSqueezeReshapePattern	UnsqueezeOrSqueezeReshape	0	Replaces the sequence Unsqueeze or Unsqueeze, Reshape by Reshape. Model with nodes to be fused…
83	UnsqueezeReshapePattern	UnsqueezeReshape	0	Unsqueeze + Reshape into Unsqueeze at a different place if possible. Model with nodes to be fused…
84	UnsqueezeUnsqueezePattern	UnsqueezeUnsqueeze	0	Replaces the sequence Unsqueeze, Unsqueeze by Unsqueeze. Model with nodes to be fused…
85	WhereAddPattern	WhereAdd	1	Replaces the sequence Add(X, Where(bool_mask, 0, -inf)) -> Where(bool_mask, X, -inf). Model with nodes to be fused…
86	RotaryConcatPartPattern	RotaryConcatPart	1	Optimizes the following pattern
87	FunctionAttentionPattern	FunctionAttention	0	Merges Attention nodes into a local function. That includes a version for GroupQueryAttention (see second pattern). Main Pattern ++++++++++++
88	FunctionAttentionGQAPattern	FunctionAttentionGQA	0	Merges onnx nodes equivalent to repeat interleave followed by function `LocalAttention` into `LocalAttentionGQA` (GQA for GroupQueryAttention). Model with nodes to be fused…
89	FunctionCausalMaskPattern	FunctionCausalMask	1	Fuses nodes matching CausalMask into a local function.
90	FunctionCausalMaskMulAddPattern	FunctionCausalMaskMulAdd	1	Fuses nodes matching CausalMask into a local function.
91	FunctionCosSinCachePattern	FunctionCosSinCache	1	Fuses nodes to simplify the creation of cos/sin caches in LLM.
92	FunctionHalfRotaryEmbeddingPattern	FunctionHalfRotaryEmbedding	1	Fuses nodes matching half RotaryEmbedding(23) into a local function.
93	RMSNormalizationPattern	RMSNormalization	1	Fuses the nodes equivalent to RMSNormalization(23). Model with nodes to be fused…
94	RMSNormalizationMulPattern	RMSNormalizationMul	1	Fuses the nodes RMSNormalization(23) + Mul into RMSNormalization. Model with nodes to be fused…
95	AttentionGQAPattern	AttentionGQA	2	Fuses LocalAttention into Attention. Opset must be >= 23 to do so.