Compares the conversions of the same model with different options

The script compares two onnx models obtained with the same trained scikit-learn models but converted with different options.

A model

from sklearn.mixture import GaussianMixture
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from skl2onnx import to_onnx
from onnx_array_api.reference import compare_onnx_execution
from onnx_array_api.plotting.text_plot import onnx_simple_text_plot


data = load_iris()
X_train, X_test = train_test_split(data.data)
model = GaussianMixture()
model.fit(X_train)

GaussianMixture()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Conversion to onnx

onx = to_onnx(
    model, X_train[:1], options={id(model): {"score_samples": True}}, target_opset=12
)

print(onnx_simple_text_plot(onx))

opset: domain='' version=12
input: name='X' type=dtype('float64') shape=['', 4]
init: name='Ad_Addcst' type=float64 shape=(1,) -- array([7.35150827])
init: name='Ge_Gemmcst' type=float64 shape=(4, 4)
init: name='Ge_Gemmcst1' type=float64 shape=(4,) -- array([-7.51792282, -7.89910213,  4.37651563,  3.02521798])
init: name='Mu_Mulcst' type=float64 shape=(1,) -- array([-0.5])
init: name='Ad_Addcst1' type=float64 shape=(1,) -- array([3.24465589])
init: name='Ad_Addcst2' type=float64 shape=(1,) -- array([0.])
Gemm(X, Ge_Gemmcst, Ge_Gemmcst1, alpha=1.00, beta=1.00) -> Ge_Y0
  ReduceSumSquare(Ge_Y0, axes=[1], keepdims=1) -> Re_reduced0
    Concat(Re_reduced0, axis=1) -> Co_concat_result0
      Add(Ad_Addcst, Co_concat_result0) -> Ad_C02
        Mul(Ad_C02, Mu_Mulcst) -> Mu_C0
          Add(Mu_C0, Ad_Addcst1) -> Ad_C01
            Add(Ad_C01, Ad_Addcst2) -> Ad_C0
              ArgMax(Ad_C0, axis=1) -> label
              ReduceLogSumExp(Ad_C0, axes=[1], keepdims=1) -> score_samples
              Sub(Ad_C0, score_samples) -> Su_C0
                Exp(Su_C0) -> probabilities
output: name='label' type=dtype('int64') shape=['', 1]
output: name='probabilities' type=dtype('float64') shape=['', 1]
output: name='score_samples' type=dtype('float64') shape=['', 1]

Conversion to onnx without ReduceLogSumExp

onx2 = to_onnx(
    model,
    X_train[:1],
    options={id(model): {"score_samples": True}},
    black_op={"ReduceLogSumExp"},
    target_opset=12,
)

print(onnx_simple_text_plot(onx2))

opset: domain='' version=12
input: name='X' type=dtype('float64') shape=['', 4]
init: name='Ad_Addcst' type=float64 shape=(1,) -- array([7.35150827])
init: name='Ge_Gemmcst' type=float64 shape=(4, 4)
init: name='Ge_Gemmcst1' type=float64 shape=(4,) -- array([-7.51792282, -7.89910213,  4.37651563,  3.02521798])
init: name='Mu_Mulcst' type=float64 shape=(1,) -- array([-0.5])
init: name='Ad_Addcst1' type=float64 shape=(1,) -- array([3.24465589])
init: name='Ad_Addcst2' type=float64 shape=(1,) -- array([0.])
Gemm(X, Ge_Gemmcst, Ge_Gemmcst1, alpha=1.00, beta=1.00) -> Ge_Y0
  Mul(Ge_Y0, Ge_Y0) -> Mu_C01
    ReduceSum(Mu_C01, axes=[1], keepdims=1) -> Re_reduced0
      Concat(Re_reduced0, axis=1) -> Co_concat_result0
        Add(Ad_Addcst, Co_concat_result0) -> Ad_C02
          Mul(Ad_C02, Mu_Mulcst) -> Mu_C0
            Add(Mu_C0, Ad_Addcst1) -> Ad_C01
              Add(Ad_C01, Ad_Addcst2) -> Ad_C0
                ArgMax(Ad_C0, axis=1) -> label
                ReduceMax(Ad_C0, axes=[1], keepdims=1) -> Re_reduced03
                Sub(Ad_C0, Re_reduced03) -> Su_C01
                  Exp(Su_C01) -> Ex_output0
                    ReduceSum(Ex_output0, axes=[1], keepdims=1) -> Re_reduced02
                      Log(Re_reduced02) -> Lo_output0
                  Add(Lo_output0, Re_reduced03) -> score_samples
                Sub(Ad_C0, score_samples) -> Su_C0
                  Exp(Su_C0) -> probabilities
output: name='label' type=dtype('int64') shape=['', 1]
output: name='probabilities' type=dtype('float64') shape=['', 1]
output: name='score_samples' type=dtype('float64') shape=['', 1]

Differences

Function onnx_array_api.reference.compare_onnx_execution() compares the intermediate results of two onnx models. Then it finds the best alignmet between the two models using an edit distance.

res1, res2, align, dc = compare_onnx_execution(onx, onx2, verbose=1)
print("------------")
text = dc.to_str(res1, res2, align)
print(text)

[compare_onnx_execution] generate inputs
[compare_onnx_execution] execute with 1 inputs
[compare_onnx_execution] execute first model
[compare_onnx_execution] got 21 results
[compare_onnx_execution] execute second model
[compare_onnx_execution] got 21 results (first model)
[compare_onnx_execution] got 27 results (second model)
[compare_onnx_execution] compute edit distance
[compare_onnx_execution] got 27 pairs
[compare_onnx_execution] done
------------
001 = | INITIA float64  1:1                  HAAA                 Ad | INITIA float64  1:1                  HAAA                 Ad
002 = | INITIA float64  2:4x4                ADZF                 Ge | INITIA float64  2:4x4                ADZF                 Ge
003 = | INITIA float64  1:4                  TTED                 Ge | INITIA float64  1:4                  TTED                 Ge
004 = | INITIA float64  1:1                  AAAA                 Mu | INITIA float64  1:1                  AAAA                 Mu
005 = | INITIA float64  1:1                  DAAA                 Ad | INITIA float64  1:1                  DAAA                 Ad
006 = | INITIA float64  1:1                  AAAA                 Ad | INITIA float64  1:1                  AAAA                 Ad
007 = | INPUT  float64  2:1x4                AAAA                 X  | INPUT  float64  2:1x4                AAAA                 X
008 = | RESULT float64  2:1x4                TTFF Gemm            Ge | RESULT float64  2:1x4                TTFF Gemm            Ge
009 + |                                                              | RESULT float64  2:1x4                EBFD Mul             Mu
010 ~ | RESULT float64  2:1x1                PAAA ReduceSumSquare Re | RESULT float64  2:1x1                PAAA ReduceSum       Re
011 = | RESULT float64  2:1x1                PAAA Concat          Co | RESULT float64  2:1x1                PAAA Concat          Co
012 = | RESULT float64  2:1x1                XAAA Add             Ad | RESULT float64  2:1x1                XAAA Add             Ad
013 = | RESULT float64  2:1x1                PAAA Mul             Mu | RESULT float64  2:1x1                PAAA Mul             Mu
014 = | RESULT float64  2:1x1                SAAA Add             Ad | RESULT float64  2:1x1                SAAA Add             Ad
015 = | RESULT float64  2:1x1                SAAA Add             Ad | RESULT float64  2:1x1                SAAA Add             Ad
016 = | RESULT int64    2:1x1                AAAA ArgMax          la | RESULT int64    2:1x1                AAAA ArgMax          la
017 + |                                                              | RESULT float64  2:1x1                SAAA ReduceMax       Re
018 + |                                                              | RESULT float64  2:1x1                AAAA Sub             Su
019 + |                                                              | RESULT float64  2:1x1                BAAA Exp             Ex
020 + |                                                              | RESULT float64  2:1x1                BAAA ReduceSum       Re
021 + |                                                              | RESULT float64  2:1x1                AAAA Log             Lo
022 ~ | RESULT float64  2:1x1                SAAA ReduceLogSumExp sc | RESULT float64  2:1x1                SAAA Add             sc
023 = | RESULT float64  2:1x1                AAAA Sub             Su | RESULT float64  2:1x1                AAAA Sub             Su
024 = | RESULT float64  2:1x1                BAAA Exp             pr | RESULT float64  2:1x1                BAAA Exp             pr
025 = | OUTPUT int64    2:1x1                AAAA                 la | OUTPUT int64    2:1x1                AAAA                 la
026 = | OUTPUT float64  2:1x1                BAAA                 pr | OUTPUT float64  2:1x1                BAAA                 pr
027 = | OUTPUT float64  2:1x1                SAAA                 sc | OUTPUT float64  2:1x1                SAAA                 sc

See onnx_array_api.reference.compare_onnx_execution for a better view. The display shows that ReduceSumSquare was replaced by Mul + ReduceSum, and ReduceLogSumExp by ReduceMax + Sub + Exp + Log + Add.