.xoptim.patterns_investigation.llm_patterns

class experimental_experiment.xoptim.patterns_investigation.llm_patterns.FunctionPackedMatMulPattern(verbose: int = 0, priority: int = 1, min_opset: int = 1)

Replaces multiple MatMul (X,A), (X,B) by (X, concat(A,B))…

apply(g: GraphBuilderPatternOptimization, *nodes: NodeProto) → List[NodeProto]

The method does the rewriting. It assumes it can happen. It takes a list of nodes impacted by the rewriting assumes no other pattern optimizer will be modify them. It receives the list of nodes returned by method apply. Since it is a list of argument, method match can include None values. The method returns the new nodes. The optimizer considers that any node given to this function is removed from the graph, and any node returned by it are added. If a received node must be kept, it must be added to the list of returned node.

Parameters:

nodes – nodes returned by method match, there are then removed

Returns:

nodes to add to graph.

match(g: GraphBuilderPatternOptimization, node: NodeProto, matched: List[MatchResult]) → MatchResult | None

Determines nodes around node which can be rewritten.

Parameters:

• g – is a GraphBuilderPatternOptimization, it holds all the existing nodes, is able to return any information about type, shape, the node before, the node after another one.

• node – the matching must determine if some nodes around this one are part of set of nodes this pattern optmizer can rewrite. From there, the function explores wherever it needs, checking any condition it needs.

• matched – usually unused, it returns of nodes already matching a pattern

The method must not modify the graph. The method returns None if no match is found or an instance of class MatchResult. It must contain:

• a list of nodes involved in the rewriting. It does not mean all of them will be removed but all of them are needed to do the rewriting and must not be impacted by other pattern optimizer.

• A function doing the rewriting (usually method apply of the pattern class).

• An existing node where the rewritten nodes can be inserted. Knowing it makes it faster to rewriter. If not specified, the optimizer will automatically determine the position of the new nodes.

class experimental_experiment.xoptim.patterns_investigation.llm_patterns.FunctionPowTanhPattern(verbose: int = 0, priority: int = 0, min_opset: int = 1)

Moves the nodes in match_pattern into a local function.

<<<

from experimental_experiment.xbuilder import GraphBuilder
from experimental_experiment.xoptim import GraphBuilderPatternOptimization
from experimental_experiment.xoptim.patterns_investigation.llm_patterns import (
    FunctionPowTanhPattern,
)

pat = FunctionPowTanhPattern()
g = GraphBuilderPatternOptimization(GraphBuilder(18))
print(pat._pattern_to_string(g))

>>>

        match_pattern(X, three, o_o_four, half, o_height, one) -> _onx_mul_mul_X
        Pow(X, three) -> _onx_pow_X
        Mul(_onx_pow_X, o_o_four) -> _onx_mul_pow_X
        Add(X, _onx_mul_pow_X) -> _onx_add_X
        Mul(X, half) -> _onx_mul_X
        Mul(_onx_add_X, o_height) -> _onx_mul_add_X
        Tanh(_onx_mul_add_X) -> _onx_tanh_mul_add_X
        Add(_onx_tanh_mul_add_X, one) -> _onx_add_tanh_mul_add_X
        Mul(_onx_mul_X, _onx_add_tanh_mul_add_X) -> _onx_mul_mul_X

apply_pattern(g, X, three, o_o_four, half, o_height, one)

Applies the replacement.

match_pattern(g: GraphBuilder, X, three, o_o_four, half, o_height, one)

Builds the pattern to match.

class experimental_experiment.xoptim.patterns_investigation.llm_patterns.FunctionSplitRotaryMulPattern(verbose: int = 0, priority: int = 0, min_opset: int = 1)

Moves the nodes in match_pattern into a local function. This is only applicable when the pattern to match has more than one node.

<<<

from experimental_experiment.xbuilder import GraphBuilder
from experimental_experiment.xoptim import GraphBuilderPatternOptimization
from experimental_experiment.xoptim.patterns_investigation.llm_patterns import (
    FunctionSplitRotaryMulPattern,
)

pat = FunctionSplitRotaryMulPattern()
g = GraphBuilderPatternOptimization(GraphBuilder(18))
print(pat._pattern_to_string(g))

>>>

        match_pattern(X, split1, split2, C1, C2) -> _onx_concat_add_mul_split_X_0
        Split(X, split1) -> _onx_split_X_0, _onx_split_X_1
        Split(_onx_split_X_0, split2) -> _onx_split_split_X_0_0, _onx_split_split_X_0_1
        Neg(_onx_split_split_X_0_1) -> _onx_neg_split_split_X_0_1
        Concat(_onx_neg_split_split_X_0_1, _onx_split_split_X_0_0) -> _onx_concat_neg_split_split_X_0_1
        Mul(_onx_split_X_0, C2) -> _onx_mul_split_X_0
        Mul(_onx_concat_neg_split_split_X_0_1, C1) -> _onx_mul_concat_neg_split_split_X_0_1
        Add(_onx_mul_split_X_0, _onx_mul_concat_neg_split_split_X_0_1) -> _onx_add_mul_split_X_0
        Concat(_onx_add_mul_split_X_0, _onx_split_X_1) -> _onx_concat_add_mul_split_X_0

apply_pattern(g, X, split1, split2, C1, C2)

Applies the replacement.

match_pattern(g: GraphBuilder, X, split1, split2, C1, C2)

Builds the pattern to match.