Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
experimental-experiment 0.1.0 documentation
Logo
experimental-experiment 0.1.0 documentation

Contents

  • Design
    • Custom Exporter
    • Pattern Optimizer
    • Dynamo Backends
  • Tutorial
    • Unexpected Errors
    • Start from a docker
  • Supported Model Signatures
    • Exported Programs with Static Shapes
    • Exported Programs with Dynamic Shapes
    • Exported into ONNX with Static Shapes
    • Exported into ONNX with Dynamic Shapes
  • API
    • .gradient
      • .gradient.ops
        • .gradient.ops.op_broadcast_gradient_args
      • .gradient.grad_helper
      • .gradient.loss_helper
    • .reference
      • .reference.ops
        • .reference.ops.op_add_add_mul_mul
        • .reference.ops.op_average_pool_grad
        • .reference.ops.op_cast_like
        • .reference.ops.op_concat
        • .reference.ops.op_constant_of_shape
        • .reference.ops.op_fused_matmul
        • .reference.ops.op_gather_grad
        • .reference.ops.op_memcpy_host
        • .reference.ops.op_mul_sigmoid
        • .reference.ops.op_negxplus1
        • .reference.ops.op_quick_gelu
        • .reference.ops.op_replace_zero
        • .reference.ops.op_rotary
        • .reference.ops.op_qlinear_average_pool
        • .reference.ops.op_qlinear_conv
        • .reference.ops.op_scatter_elements
        • .reference.ops.op_scatternd_of_shape
        • .reference.ops.op_simplified_layer_normalization
        • .reference.ops.op_skip_layer_normalization
        • .reference.ops.op_slice
        • .reference.ops.op_transpose_cast
        • .reference.ops.op_tri_matrix
      • .reference.evaluator
      • .reference.ort_evaluator
      • .reference.quantized_tensor
    • .convert
      • .convert.convert_helper
      • .convert.ort_helper
    • .plotting
      • .plotting.data
      • .plotting.memory
    • .skl
      • .skl.convert
      • .skl.helpers
    • .torch_interpreter
      • .torch_interpreter._aten_functions
      • .torch_interpreter._aten_functions_attention
      • .torch_interpreter._aten_methods
      • .torch_interpreter._doc_
      • .torch_interpreter._exceptions
      • .torch_interpreter._prims_functions
      • .torch_interpreter._torch_helper
      • .torch_interpreter.aten_functions
      • .torch_interpreter.aten_methods
      • .torch_interpreter.dispatcher
      • .torch_interpreter.eval
        • .torch_interpreter.eval.model_cases
      • .torch_interpreter.export_options
      • .torch_interpreter.interpreter
      • .torch_interpreter.investigate_helper
      • .torch_interpreter.onnx_export
      • .torch_interpreter.onnx_export_errors
      • .torch_interpreter.onnx_export_serialization
      • .torch_interpreter.oxs_dispatcher
      • .torch_interpreter.oxs_opset
      • .torch_interpreter.piece_by_piece
      • .torch_interpreter.piece_by_piece_serialize
      • .torch_interpreter.patches
        • .torch_interpreter.patches.patch_torch
        • .torch_interpreter.patches.patch_transformers
      • .torch_interpreter.tracing
    • .torch_models
      • .torch_models.diffusion_model_helper
      • .torch_models.dump_helper
      • .torch_models.llama_helper
      • .torch_models.llm_model_helper
      • .torch_models.llm_model_setup
      • .torch_models.mistral_helper
      • .torch_models.phi3_helper
      • .torch_models.phi_helper
      • .torch_models.training_helper
    • .xbuilder
      • .xbuilder._graph_builder_runtime
      • .xbuilder._onnx_helper
      • .xbuilder._shape_helper
      • .xbuilder.expression_dimension
      • .xbuilder.graph_builder
      • .xbuilder.graph_builder_opset
      • .xbuilder.model_container
      • .xbuilder.optimization_options
      • .xbuilder.reverse_graph_builder
      • .xbuilder.shape_type_compute
      • .xbuilder.type_inference
    • .xoptim
      • .xoptim.patterns_investigation
        • .xoptim.patterns_investigation.element_wise
        • .xoptim.patterns_investigation.llm_patterns
      • .xoptim.patterns_ml
        • .xoptim.patterns_ml.tree_ensemble
      • .xoptim.patterns_exp
        • .xoptim.patterns_exp.binary_operators
        • .xoptim.patterns_exp.constant_of_shape_scatter_nd
        • .xoptim.patterns_exp.constants
        • .xoptim.patterns_exp.simple_rotary
        • .xoptim.patterns_exp.unary_operators
        • .xoptim.patterns_exp.where_replace
      • .xoptim.patterns
        • .xoptim.patterns.onnx_any
        • .xoptim.patterns.onnx_cast
        • .xoptim.patterns.onnx_clip
        • .xoptim.patterns.onnx_constants
        • .xoptim.patterns.onnx_conv
        • .xoptim.patterns.onnx_dropout
        • .xoptim.patterns.onnx_equal
        • .xoptim.patterns.onnx_expand
        • .xoptim.patterns.onnx_functions
        • .xoptim.patterns.onnx_layer_normalization
        • .xoptim.patterns.onnx_matmul
        • .xoptim.patterns.onnx_mul
        • .xoptim.patterns.onnx_reduce
        • .xoptim.patterns.onnx_reshape
        • .xoptim.patterns.onnx_rotary
        • .xoptim.patterns.onnx_slice
        • .xoptim.patterns.onnx_split
        • .xoptim.patterns.onnx_sub
        • .xoptim.patterns.onnx_sequence
        • .xoptim.patterns.onnx_transpose
        • .xoptim.patterns.onnx_unsqueeze
      • .xoptim.patterns_ort
        • .xoptim.patterns_ort.activation
        • .xoptim.patterns_ort.activation_grad
        • .xoptim.patterns_ort.batch_normalization
        • .xoptim.patterns_ort.fused_conv
        • .xoptim.patterns_ort.fused_matmul
        • .xoptim.patterns_ort.gather_grad
        • .xoptim.patterns_ort.llm_optim
        • .xoptim.patterns_ort.simplified_layer_normalization
      • .xoptim.patterns_fix
        • .xoptim.patterns_fix.add_reduction_scatter_nd
      • .xoptim.graph_builder_optim
      • .xoptim.order_optim
      • .xoptim.patterns_api
      • .xoptim.unfused
    • .torch_dynamo
      • .torch_dynamo._dynamo_exporter
      • .torch_dynamo.backend_helper
      • .torch_dynamo.debug_backend
      • .torch_dynamo.fast_backend
      • experimental_experiment.torch_dynamo.partition
    • .torch_bench
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner_agg
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner_agg_helper
      • experimental_experiment.torch_bench._bash_bench_cmd
      • experimental_experiment.torch_bench._bash_bench_model_runner
      • experimental_experiment.torch_bench._bash_bench_models_helper
      • experimental_experiment.torch_bench._bash_bench_set_dummies
      • experimental_experiment.torch_bench._bash_bench_set_explicit
      • experimental_experiment.torch_bench._bash_bench_set_huggingface
      • experimental_experiment.torch_bench._bash_bench_set_huggingface_big
      • experimental_experiment.torch_bench._bash_bench_set_issues
      • experimental_experiment.torch_bench._bash_bench_set_timm
      • experimental_experiment.torch_bench._bash_bench_set_torchbench
      • experimental_experiment.torch_bench._bash_bench_set_torchbench_ado
      • experimental_experiment.torch_bench._bash_bench_suites
      • experimental_experiment.torch_bench._dort_cmd_common
      • experimental_experiment.torch_bench._dort_cmd_common_models
      • .torch_bench.bash_bench_agg
      • .torch_bench.bash_bench_explicit
      • .torch_bench.bash_bench_huggingface
      • .torch_bench.bash_bench_huggingface_big
      • .torch_bench.bash_bench_issues
      • .torch_bench.bash_bench_timm
      • .torch_bench.bash_bench_torchbench
      • .torch_bench.bash_bench_torchbench_ado
      • .torch_bench.bash_bench_untrained
      • .torch_bench.check_model
      • .torch_bench.dort_bench
      • .torch_bench.dort_bench_profile
      • .torch_bench.dort_profile
      • .torch_bench.export_model
      • .torch_bench.export_model_helper
    • ._bench_test
    • ._command_lines_parser
    • .args
    • .bench_run
    • .checks
    • .ext_test_case
    • .helpers
    • .memory_peak
    • .mini_onnx_builder
    • .model_run
    • .onnx_tools
    • .torch_test_helper
  • Galleries of Examples and Recipes
    • Examples Gallery
      • 101: A custom backend for torch
      • 101: Linear Regression and export to ONNX
      • 101: Onnx Model Optimization based on Pattern Rewriting
      • 101: Onnx Model Rewriting
      • 101: Profile an existing model with onnxruntime
      • 101: Some dummy examples with torch.export.export
      • 102: Convolution and Matrix Multiplication
      • 102: Examples with onnxscript
      • 102: Fuse kernels in a small Llama Model
      • 102: Measure LLAMA speed
      • 102: Tweak onnx export
      • 201: Evaluate DORT
      • 201: Evaluate DORT Training
      • 201: Evaluate different ways to export a torch model to ONNX
      • 201: Use torch to export a scikit-learn model into ONNX
      • 301: Compares LLAMA exporters
      • 301: Compares LLAMA exporters for onnxrt backend
    • Exporter Recipes Gallery
      • A dynamic dimension lost by torch.export.export
      • Do no use Module as inputs!
      • Export Phi-3.5-mini-instruct piece by piece
      • Export Phi-3.5-mini-instruct with draft_export
      • Export Phi-3.5-mini-instruct with report_exportability
      • Export a model using a custom type as input
      • Export a model with a loop (scan)
      • Infer dynamic shapes before exporting
      • Linear Regression and export to ONNX
      • Measures the exporter success on many test cases
      • Use DYNAMIC or AUTO when dynamic shapes has constraints
      • to_onnx and Phi-2
      • to_onnx and a custom operator inplace
      • to_onnx and a custom operator registered with a function
      • to_onnx and a model with a loop (scan)
      • to_onnx and a model with a test
      • to_onnx and padding one dimension to a mulitple of a constant
      • to_onnx and submodules from LLMs
      • to_onnx: Rename Dynamic Shapes
      • torch.onnx.export and Phi-2
      • torch.onnx.export and a model with a test
      • torch.onnx.export and padding one dimension to a mulitple of a constant
      • torch.onnx.export: Rename Dynamic Shapes
  • Command Lines
    • Benchmarks from the command line
      • experimental_experiment.torch_bench.dort_bench
      • experimental_experiment.torch_bench.dort_profile
      • Interesting scripts or command lines
      • Measuring the exporters on a short list of sets of models
    • Tools from the command line
      • python -m experimental_experiment lighten and unlighten
      • python -m experimental_experiment optimize
      • python -m experimental_experiment print
      • python -m experimental_experiment run
  • Miscellaneous
    • Export Times
    • Long Outputs uneasy to read
    • Supported Models By the Custom Backend
      • Phi

More

  • Change Logs
  • License
Back to top
View this page

Examples Gallery¶

A couple of examples to illustrate different implementation of dot product (see also sphinx-gallery).

Getting started¶

pytorch nightly build should be installed, see Start Locally.

git clone https://github.com/sdpython/experimental-experiment.git
pip install onnxruntime-gpu pynvml
pip install -r requirements-dev.txt
export PYTHONPATH=$PYTHONPATH:<this folder>

Compare torch exporters¶

The script evaluates the memory peak, the computation time of the exporters. It also compares the exported models when run through onnxruntime. The full script takes around 20 minutes to complete. It stores on disk all the graphs, the data used to draw them, and the models.

python _doc/examples/plot_torch_export.py -s large

See Deeper into pytorch and onnx for an organized version of this page.

101: A custom backend for torch

101: A custom backend for torch

101: Linear Regression and export to ONNX

101: Linear Regression and export to ONNX

101: Onnx Model Optimization based on Pattern Rewriting

101: Onnx Model Optimization based on Pattern Rewriting

101: Onnx Model Rewriting

101: Onnx Model Rewriting

101: Profile an existing model with onnxruntime

101: Profile an existing model with onnxruntime

101: Some dummy examples with torch.export.export

101: Some dummy examples with torch.export.export

102: Convolution and Matrix Multiplication

102: Convolution and Matrix Multiplication

102: Examples with onnxscript

102: Examples with onnxscript

102: Fuse kernels in a small Llama Model

102: Fuse kernels in a small Llama Model

102: Measure LLAMA speed

102: Measure LLAMA speed

102: Tweak onnx export

102: Tweak onnx export

201: Evaluate DORT

201: Evaluate DORT

201: Evaluate DORT Training

201: Evaluate DORT Training

201: Evaluate different ways to export a torch model to ONNX

201: Evaluate different ways to export a torch model to ONNX

201: Use torch to export a scikit-learn model into ONNX

201: Use torch to export a scikit-learn model into ONNX

301: Compares LLAMA exporters

301: Compares LLAMA exporters

301: Compares LLAMA exporters for onnxrt backend

301: Compares LLAMA exporters for onnxrt backend

Download all examples in Python source code: auto_examples_python.zip

Download all examples in Jupyter notebooks: auto_examples_jupyter.zip

Gallery generated by Sphinx-Gallery

Next
101: A custom backend for torch
Previous
Galleries of Examples and Recipes
Copyright © 2023-2024
Made with Sphinx and @pradyunsg's Furo
On this page
  • Examples Gallery
    • Getting started
    • Compare torch exporters