yobx.xoptim.patterns#

modules

get_default_patterns#

yobx.xoptim.patterns.get_default_patterns(verbose: int = 0) → List[PatternOptimization][source]#

Returns a default list of optimization patterns. It is equal to the following list.

<<<

from yobx.xoptim.patterns_api import pattern_table_doc
from yobx.xoptim.patterns import get_default_patterns

print(pattern_table_doc(get_default_patterns(), as_rst=True))

>>>

	name	short_name	priority	doc
0	BatchNormalizationPattern	BatchNormalization	0	Checks that a BatchNormalization is really needed. Model with nodes to be fused…
1	BatchNormalizationTrainingPattern	BatchNormalizationTraining	0	Checks that a BatchNormalization in training mode can be avoided.
2	CastLayerNormalizationCastPattern	CastLayerNormalizationCast	1	Checks that a Cast is really needed around GroupNormalization, LayerNormalization, RMSNormalization. Model with nodes to be fused…
3	CastPattern	Cast	0	Checks that a Cast is really needed. Model with nodes to be fused…
4	CastCastBinaryPattern	CastCastBinary	1	Moves two cast operators beyond a binary operator The cast must cast from a float type to another float type. Model with nodes to be fused…
5	CastCastPattern	CastCast	0	Checks that two consecutive cast can be avoided. Model with nodes to be fused…
6	CastOpCastPattern	CastOpCast	1	Removes two cast surrounding another operator. Model with nodes to be fused…
7	ClipClipPattern	ClipClip	1	Merges consecutive clips if one is defining min and the other max. Model with nodes to be fused…
8	ConcatEmptyPattern	ConcatEmpty	1	Checks if one of the concatenated values is empty. Model with nodes to be fused…
9	ConcatGatherPattern	ConcatGather	0	Simplifies `Gather(Concat(...), cst_index)` when the index is a constant. The Concat inputs may each have one or more elements along axis 0. The pattern locates which Concat input contains the element addressed by the constant index and replaces the Gather with: * an `Identity` node when the located input has exactly one element, or * a `Gather` node on that input with an adjusted (local) index when the input has more than one element. Model with nodes to be fused…
10	ConcatReshapePattern	ConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused…
11	ConcatTwiceUnaryPattern	ConcatTwiceUnary	1	Sin(Concat(x,x)) -> Concat(Sin(x), Sin(x)). Model with nodes to be fused…
12	ConstantToInitializerPattern	ConstantToInitializer	1	Replaces a node Constant by an initializer and a node Identity. Model with nodes to be fused…
13	ConvBiasNullPattern	ConvBiasNull	0	Checks that a Conv has a null bias. Model with nodes to be fused…
14	PadConvPattern	PadConv	0	Fuses a Pad node followed by a Conv node into a single Conv node with the padding folded into the `pads` attribute. The fusion is valid when: * The Pad mode is `constant` (default). * The constant padding value is `0` (default). * The padding is applied only to spatial dimensions (batch and channel dimensions must have zero padding). * The Conv does not already use `auto_pad` (other than `NOTSET`). Model with nodes to be fused…
15	DropoutPattern	Dropout	1	Checks that a Cast is really needed. Model with nodes to be fused…
16	ExpandPattern	Expand	0	Checks that a Expand is really needed. Model with nodes to be fused…
17	ExpandBroadcastPattern	ExpandBroadcast	1	Checks that a Expand is really needed before an element wise operator. The objective is to save one allocation and let the next operator do the expansion by broadcasting one input. Model with nodes to be fused…
18	ExpandSwapPattern	ExpandSwap	1	Tries to move a node Expand forward in the graph. Expand + Exp can be changed into Exp + Expand. Then Exp applies on a tensor of a smaller or equal size. Model with nodes to be fused…
19	ExpandUnsqueezeExpandPattern	ExpandUnsqueezeExpand	0	Fuses the sequence Expand + Unsqueeze + Expand into Unsqueeze + Expand. Since Expand does not change the rank of a tensor, the Unsqueeze axes are valid for the original tensor as well, and the final Expand can handle both the broadcasting of the first Expand and the new dimension added by Unsqueeze. Model with nodes to be fused…
20	GatherConcatPattern	GatherConcat	0	Simplifies `Gather(Concat(..., X, ..., axis=0), cst1)` into `Gather(X, cst2)` where X is a 1D tensor and cst1, cst2 are 0D or 1D integer tensors. This applies when: - The `Concat` axis is 0. - Exactly one `Concat` input (`X`) is not a constant; all others are constants with known sizes. - `cst1` is a constant 0D or 1D `INT64` tensor with non-negative values. - All indices in `cst1` fall within `X`’s slice of the concatenated tensor. The adjusted index is `cst2 = cst1 - offset` where `offset` is the sum of the sizes of all constant inputs that precede `X` in the `Concat` input list. Model with nodes to be fused…
21	GatherGatherPattern	GatherGather	0	Simplifies two consecutive Gather operations into one. Model with nodes to be fused…
22	GathersSplitPattern	GathersSplit	1	Merges multiple parallel gather into a split followed by unsqueeze. Model with nodes to be fused…
23	GatherShapePattern	GatherShape	0	Simplifies `Gather(Shape(X), indices)` into `Shape(X, start=s, end=e)` when indices is a constant 1-D `int64` array that forms a contiguous ascending range `[s, s+1, ..., e-1]`. This avoids materialising the full shape vector only to slice it immediately afterwards. The Shape node may already carry `start` / `end` attributes (ONNX opset ≥ 15); those are taken into account when computing the absolute indices in `X`’s dimension space. Model with nodes to be fused…
24	GeluPattern	Gelu	0	Detects the decomposed version of Gelu with Tanh
25	IdentityPattern	Identity	0	Replaces operator such as Div(X, 1), Mul(X, 1), Add(X, 0), Sub(X, 0), Transpose(X, [0, 1, 2, …]), or Reshape(X, [0, 0, …]) by identity nodes. It looks into patterns involving the following operators…
26	LayerNormalizationPattern	LayerNormalization	1	Fuses nodes of a LayerNormalization. Model with nodes to be fused…
27	LayerNormalizationScalePattern	LayerNormalizationScale	1	Fused LayerNormalization, scale, bias just after. Model with nodes to be fused…
28	LeakyReluPattern	LeakyRelu	0	Detects the decomposed version of LeakyRelu. Model with nodes to be fused…
29	MaxReluPattern	MaxRelu	1	Replaces `Max(x, 0)` or `Max(0, x)` with `Relu(x)`. Model with nodes to be fused…
30	MulMulMulScalarPattern	MulMulMulScalar	1	Replaces the sequence {Div \| Mul} and {Div \| Mul} + {Div \| Mul} with {Div \| Mul} Mul. Model with nodes to be fused…
31	MulUnsqueezeUnsqueezePattern	MulUnsqueezeUnsqueeze	0	Replaces `Mul(Unsqueeze(x, axes), Unsqueeze(y, axes))` by `Unsqueeze(Mul(x, y), axes)` when both inputs are unsqueezed with the same axes. Model with nodes to be fused…
32	NotNotPattern	NotNot	1	Fuses Not + Not into Identity. Model with nodes to be fused…
33	NotWherePattern	NotWhere	1	Replaces the sequence Where(Not(cond), X, Y) -> Where(cond, Y, X).
34	ReduceArgTopKPattern	ReduceArgTopK	1	Fuses ReduceMin(X, axis), ArgMin(X, axis) into TopK(, k=1). Model with nodes to be fused…
35	ReduceReshapePattern	ReduceReshape	1	Replaces the sequence Reduce* Reshape if reshape is only introduces to deal with a dimension kept because keepdims=1. Model with nodes to be fused…
36	ReduceSumNormalizePattern	ReduceSumNormalize	1	Nodes equivalent to a reduction. Model with nodes to be fused…
37	ReshapePattern	Reshape	0	Checks that a Reshape is really needed.
38	ReshapeMatMulReshapePattern	ReshapeMatMulReshape	1	Replaces the sequence Reshape, Matmul, Reshape by Matmul. Model with nodes to be fused…
39	Reshape2Of3Pattern	Reshape2Of3	1	Replaces the reshapes around element-wise operators. It can be 3 or 2 out of 3. Model with nodes to be fused…
40	ReshapeReshapeBinaryPattern	ReshapeReshapeBinary	1	Moves two reshape operators beyond a binary operator if it is possible. Model with nodes to be fused…
41	ReshapeSqueezePattern	ReshapeSqueeze	0	Replaces the sequence Reshape, Squeeze by a single Reshape when the Reshape introduces a size-1 dimension that is then removed by the Squeeze. Model with nodes to be fused…
42	MatMulAddPattern	MatMulAdd	3	Replaces the sequence MatMul, Add into Gemm. By default, no reshape is allowed this happens only it is two dimensions. Model with nodes to be fused…
43	GemmTransposePattern	GemmTranspose	1	Replaces Gemm (., constant) by Gemm(., constant’, transB=1) Model with nodes to be fused…
44	MatMulReshape2Of3Pattern	MatMulReshape2Of3	1	Replaces the reshapes around a matmul It can be 3 or 2 out of 3. It is similar to `yobx.xoptim.patterns.onnx_reshape.Reshape2Of3Pattern`. Model with nodes to be fused…
45	MulMulMatMulPattern	MulMulMatMul	1	Replaces `MatMul(ac, bd)` where c and d are constant scalar by `MatMul(a,b) * (c,d)`. Model with nodes to be fused…
46	ShapeBasedReshapeIsSqueezePattern	ShapeBasedReshapeIsSqueeze	0	Replaces a Reshape by a Squeeze or Unsqueeze pattern if possible. It is only available for `opset >= 18`. Model with nodes to be fused…
47	ShapeBasedStaticExpandPattern	ShapeBasedStaticExpand	0	Compares input and output shapes to tell if the expand can uses a constant as a second input. Model with nodes to be fused…
48	ShapeBasedConcatExpandPattern	ShapeBasedConcatExpand	1	Rewrites Expand(X, concat(…)) if possible. Model with nodes to be fused…
49	ShapeBasedEditDistanceReshapePattern	ShapeBasedEditDistanceReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1 or 0. The pattern tries to align shape information to infer a static shape. Model with nodes to be fused…
50	ShapeBasedIdentityPattern	ShapeBasedIdentity	0	If a slice leads to the same shape and the step is 1 then it is identity. In some cases, just known the same is enough to replace them. Model with nodes to be fused…
51	ShapeBasedExpandBroadcastPattern	ShapeBasedExpandBroadcast	1	Similar to `yobx.xoptim.patterns.onnx_expand.ExpandBroadcastPattern`, but it allows dynamic shapes as well. It does not look into the second argument of Expand, it just infers than an expand is not needed for a binary operator following just after. Model with nodes to be fused…
52	ShapeBasedExpandBroadcastMatMulPattern	ShapeBasedExpandBroadcastMatMul	1	Similar to `yobx.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern`, but works only with MatMul. Model with nodes to be fused…
53	ShapeBasedExpandCastWhereSwapPattern	ShapeBasedExpandCastWhereSwap	1	Rewrites Where(Cast(X), X, cond). Model with nodes to be fused…
54	ShapeBasedExpandSwapPattern	ShapeBasedExpandSwap	1	Tries to move a node Expand forward in the graph for a binary operator. The code is similar to `yobx.xoptim.patterns.onnx_expand.ShapeBasedExpandBroadcastPattern` Model with nodes to be fused…
55	ShapeBasedMatMulToMulPattern	ShapeBasedMatMulToMul	1	MatMul can be replaced by Mul with broadcast. It makes it easier to detect optimization pattern with Expand operators. Model with nodes to be fused…
56	ShapedBasedReshapePattern	ShapedBasedReshape	0	Checks that a Reshape is really needed based on the input shape. Model with nodes to be fused…
57	ShapeBasedSameChildrenPattern	ShapeBasedSameChildren	0	Checks there is no duplicated node doing the same than another one beside. `yobx.xoptim.patterns.onnx_any.SameChildrenPattern` checks it is exactly the same. This one assumes it is exactly the same in some cases such expand (X, sh1) = expand(X, sh2) if the output shapes are the same. Model with nodes to be fused…
58	ShapeBasedShapeShapeAddPattern	ShapeBasedShapeShapeAdd	0	Tries to find another way to get a dimension obtained with the addition of two.
59	ShapeTransposePattern	ShapeTranspose	0	Replaces `Shape(Transpose(X, perm))` by `Gather(Shape(X), perm_indices)` so that the expensive Transpose on the full data tensor is avoided. The key observation is that the shape of `Transpose(X, perm)` is simply a permuted view of the shape of `X`. The permutation indices are known at optimisation time (they are an attribute of the Transpose node), so we can extract the desired dimensions directly from `Shape(X)` using a `Gather` with the (sub-)permutation as the index tensor. For `X` of shape `(a, b, c)` and `perm=[2, 0, 1]` the transformation is
60	UnsqueezeShapePattern	UnsqueezeShape	0	Replaces `Shape(Unsqueeze(X, axes))` by a `Concat` of `Shape(X, start=s, end=e)` slices interleaved with constant `[1]` tensors at the inserted axis positions. The key observation is that `Shape(Unsqueeze(X, axes))` produces a shape vector that is identical to `Shape(X)` with `1` entries inserted at the `axes` positions. By splitting `Shape(X)` into segments and concatenating them with the constant `1` values, the Unsqueeze on the (potentially large) data tensor is avoided entirely while the output shape vector remains bit-for-bit identical. For `X` of shape `(a, b, c)` and `axes=[1]` the transformation is
61	ReshapeReshapePattern	ReshapeReshape	0	Replaces the sequence Reshape, Reshape by Reshape. Model with nodes to be fused…
62	RotaryEmbeddingPattern	RotaryEmbedding	1	Fuses nodes matching RotaryEmbedding(23). Model with nodes to be fused…
63	SameChildrenPattern	SameChildren	0	Checks there is no duplicated node doing the same than another one beside. Model with nodes to be fused…
64	SameChildrenFromInputPattern	SameChildrenFromInput	0	Checks there is no duplicated node doing the same than another one beside and taking a model input as input. Model with nodes to be fused…
65	SequenceConstructAtPattern	SequenceConstructAt	1	Replaces the sequence `SequenceConstruct(x1, x2, ...)` followed by `SequenceAt(seq, 0)`, `SequenceAt(seq, 1)`, … Model with nodes to be fused…
66	SplitToSequenceSequenceAtPattern	SplitToSequenceSequenceAt	1	Replaces `SplitToSequence(x, split, axis=a)` followed by `SequenceAt(seq, 0)`, `SequenceAt(seq, 1)`, … into `Split(x, split, axis=a)`. When `keepdims=0`, a `Squeeze` node is added after each split output to remove the split axis dimension (which has size 1 for each chunk). Model with nodes to be fused…
67	SliceSlicePattern	SliceSlice	1	Merges consecutive slices if axis are disjoints. Model with nodes to be fused…
68	SlicesSplitPattern	SlicesSplit	1	Merges multiple parallel slices into a split. Model with nodes to be fused…
69	SoftmaxCrossEntropyLossCastPattern	SoftmaxCrossEntropyLossCast	0	Detects one decomposed version of SoftmaxCrossEntropyLoss. Model with nodes to be fused…
70	SplitConcatPattern	SplitConcat	1	Replaces Split + Concat into identity if this is equivalent. Model with nodes to be fused…
71	SqueezeAddPattern	SqueezeAdd	0	Replaces the sequence Add(Squeeze, Squeeze) by Squeeze(Add). Model with nodes to be fused…
72	SqueezeBinaryUnsqueezePattern	SqueezeBinaryUnsqueeze	0	Replaces the sequence Squeeze Binary Unsqueeze) by Binary. Model with nodes to be fused…
73	SqueezeUnsqueezePattern	SqueezeUnsqueeze	0	Replaces the sequence Squeeze, Unsqueeze by Identity or the other ways around. Model with nodes to be fused…
74	StaticConcatReshapePattern	StaticConcatReshape	0	Tries to reduce the number of nodes in the sequence Concat + Reshape by replacing one of the dimension by -1. Model with nodes to be fused…
75	Sub1MulPattern	Sub1Mul	1	Replaces the sequence (1 - X) x Y by Y - X x Y to avoid the creation of a constant in the graph. x means element wise multiplication. Model with nodes to be fused…
76	SwapExpandReshapePattern	SwapExpandReshape	0	Checks if Expand + Reshape can be swapped. Model with nodes to be fused…
77	SwapExpandUnsqueezePattern	SwapExpandUnsqueeze	0	Swaps Expand and Unsqueeze when Unsqueeze directly follows Expand. `Expand(X, shape) → Unsqueeze(expanded, axes)` is rewritten as `Unsqueeze(X, axes) → Expand(unsqueezed, new_shape)` where `new_shape` is obtained by inserting `1` at every position listed in `axes` into the original expand shape. Performing the Unsqueeze before the Expand means the Unsqueeze operates on the smaller (pre-expanded) tensor, which is more efficient. Model with nodes to be fused…
78	SwapRangeAddScalarPattern	SwapRangeAddScalar	1	Swap Range + Add when a scalar is added. Model with nodes to be fused…
79	SwapUnaryPattern	SwapUnary	0	Tries to move computation nodes before any transpose or reshape. That works for unary operator or equivalent to that. Model with nodes to be fused…
80	SwapUnsqueezeTransposePattern	SwapUnsqueezeTranspose	0	Swaps Unsqueeze and Transpose. Model with nodes to be fused…
81	SwitchOrderBinaryPattern	SwitchOrderBinary	1	If it makes sense, switches the order of two multiplications or two additions if the broadcasting reduces one operator to a an insignificant number. Model with nodes to be fused…
82	SwitchReshapeActivationPattern	SwitchReshapeActivation	1	Switches Gelu and Reshape after a Gemm or a MatMul. Gelu can also be Gelu, Exp, Elu, Relu, Tan, Tanh, Cos, Cosh, Sin, Sinh, Erf, LeakyRelu, PRelu, Selu, Softmax, Softplus. Reshape can also be Transpose. Model with nodes to be fused…
83	TransposeEqualReshapePattern	TransposeEqualReshape	1	Replaces a Transpose by a Reshape when switched dimensions are all equal to 1 but one. Model with nodes to be fused…
84	TransposeGatherPattern	TransposeGather	0	Removes one unnecessary transpose followed by Gather with only one index. Model with nodes to be fused…
85	TransposeMatMulPattern	TransposeMatMul	1	Replaces the sequence Transpose, Matmul or Gemm into Gemm Model with nodes to be fused…
86	TransposeReshapeMatMulPattern	TransposeReshapeMatMul	1	Replaces the sequence Transpose, Reshape, Matmul into Reshape, Transpose, Matmul if possible. Another optimizer will optimizes this sequence by using Gemm or better. Model with nodes to be fused…
87	TransposeReshapeTransposePattern	TransposeReshapeTranspose	0	Swaps Reshape and Transpose in a sequence such as this one…
88	TransposeTransposePattern	TransposeTranspose	0	Removes two consecutive transpose if the second one put the tensor in origin shape. Model with nodes to be fused…
89	UnsqueezeEqualPattern	UnsqueezeEqual	1	Replaces the sequence R -> Equal -> Unsqueeze, R -> Unsqueeze, into R -> Unsqueeze -> Equal. Model with nodes to be fused…
90	UnsqueezeOrSqueezeReshapePattern	UnsqueezeOrSqueezeReshape	0	Replaces the sequence Unsqueeze or Unsqueeze, Reshape by Reshape. Model with nodes to be fused…
91	UnsqueezeReshapePattern	UnsqueezeReshape	0	Unsqueeze + Reshape into Unsqueeze at a different place if possible. Model with nodes to be fused…
92	UnsqueezeUnsqueezePattern	UnsqueezeUnsqueeze	0	Replaces the sequence Unsqueeze, Unsqueeze by Unsqueeze. Model with nodes to be fused…
93	WhereAddPattern	WhereAdd	1	Replaces the sequence Add(X, Where(bool_mask, 0, -inf)) -> Where(bool_mask, X, -inf). Model with nodes to be fused…
94	RotaryConcatPartPattern	RotaryConcatPart	1	Optimizes the following pattern
95	FunctionAttentionPattern	FunctionAttention	0	Merges Attention nodes into a local function. That includes a version for GroupQueryAttention (see second pattern). Main Pattern ++++++++++++
96	FunctionAttentionGQAPattern	FunctionAttentionGQA	0	Merges onnx nodes equivalent to repeat interleave followed by function `LocalAttention` into `LocalAttentionGQA` (GQA for GroupQueryAttention). Model with nodes to be fused…
97	FunctionCausalMaskPattern	FunctionCausalMask	1	Fuses nodes matching CausalMask into a local function.
98	FunctionCausalMaskMulAddPattern	FunctionCausalMaskMulAdd	1	Fuses nodes matching CausalMask into a local function.
99	FunctionCosSinCachePattern	FunctionCosSinCache	1	Fuses nodes to simplify the creation of cos/sin caches in LLM.
100	FunctionHalfRotaryEmbeddingPattern	FunctionHalfRotaryEmbedding	1	Fuses nodes matching half RotaryEmbedding(23) into a local function.
101	RMSNormalizationPattern	RMSNormalization	1	Fuses the nodes equivalent to RMSNormalization(23). Model with nodes to be fused…
102	RMSNormalizationMulPattern	RMSNormalizationMul	1	Fuses the nodes RMSNormalization(23) + Mul into RMSNormalization. Model with nodes to be fused…
103	AttentionGQAPattern	AttentionGQA	2	Fuses LocalAttention into Attention. Opset must be >= 23 to do so.