Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
experimental-experiment 0.1.0 documentation
Logo
experimental-experiment 0.1.0 documentation

Contents

  • Design
    • Custom Exporter
    • Pattern Optimizer
    • Dynamo Backends
  • Tutorial
    • Exported Programs with Static Shapes
    • Exported Programs with Dynamic Shapes
    • Exported ONNX with Static Shapes
    • Exported ONNX with Dynamic Shapes
    • Start from a docker
  • API
    • experimental_experiment.gradient
      • experimental_experiment.gradient.ops
        • experimental_experiment.gradient.ops.op_broadcast_gradient_args
      • experimental_experiment.gradient.grad_helper
      • experimental_experiment.gradient.loss_helper
    • experimental_experiment.reference
      • experimental_experiment.reference.ops
        • experimental_experiment.reference.ops.op_add_add_mul_mul
        • experimental_experiment.reference.ops.op_average_pool_grad
        • experimental_experiment.reference.ops.op_cast_like
        • experimental_experiment.reference.ops.op_concat
        • experimental_experiment.reference.ops.op_constant_of_shape
        • experimental_experiment.reference.ops.op_fused_matmul
        • experimental_experiment.reference.ops.op_gather_grad
        • experimental_experiment.reference.ops.op_memcpy_host
        • experimental_experiment.reference.ops.op_mul_sigmoid
        • experimental_experiment.reference.ops.op_negxplus1
        • experimental_experiment.reference.ops.op_quick_gelu
        • experimental_experiment.reference.ops.op_replace_zero
        • experimental_experiment.reference.ops.op_rotary
        • experimental_experiment.reference.ops.op_scatter_elements
        • experimental_experiment.reference.ops.op_scatternd_of_shape
        • experimental_experiment.reference.ops.op_simplified_layer_normalization
        • experimental_experiment.reference.ops.op_slice
        • experimental_experiment.reference.ops.op_transpose_cast
        • experimental_experiment.reference.ops.op_tri_matrix
      • experimental_experiment.reference.evaluator
      • experimental_experiment.reference.ort_evaluator
    • experimental_experiment.convert
      • experimental_experiment.convert.convert_helper
      • experimental_experiment.convert.ort_helper
    • experimental_experiment.plotting
      • experimental_experiment.plotting.data
      • experimental_experiment.plotting.memory
    • experimental_experiment.torch_interpreter
      • experimental_experiment.torch_interpreter._aten_functions
      • experimental_experiment.torch_interpreter._aten_functions_attention
      • experimental_experiment.torch_interpreter._aten_methods
      • experimental_experiment.torch_interpreter._doc_
      • experimental_experiment.torch_interpreter._exceptions
      • experimental_experiment.torch_interpreter._prims_functions
      • experimental_experiment.torch_interpreter._torch_helper
      • experimental_experiment.torch_interpreter.aten_functions
      • experimental_experiment.torch_interpreter.aten_methods
      • experimental_experiment.torch_interpreter.dispatcher
      • experimental_experiment.torch_interpreter.eval
        • experimental_experiment.torch_interpreter.eval.model_cases
      • experimental_experiment.torch_interpreter.export_options
      • experimental_experiment.torch_interpreter.interpreter
      • experimental_experiment.torch_interpreter.onnx_export
      • experimental_experiment.torch_interpreter.onnx_export_errors
      • experimental_experiment.torch_interpreter.onnx_export_serialization
      • experimental_experiment.torch_interpreter.oxs_dispatcher
      • experimental_experiment.torch_interpreter.oxs_opset
      • experimental_experiment.torch_interpreter.patches
        • experimental_experiment.torch_interpreter.patches.patch_torch
        • experimental_experiment.torch_interpreter.patches.patch_transformers
      • experimental_experiment.torch_interpreter.tracing
    • experimental_experiment.torch_models
      • experimental_experiment.torch_models.diffusion_model_helper
      • experimental_experiment.torch_models.dump_helper
      • experimental_experiment.torch_models.llama_helper
      • experimental_experiment.torch_models.llm_model_helper
      • experimental_experiment.torch_models.llm_model_setup
      • experimental_experiment.torch_models.mistral_helper
      • experimental_experiment.torch_models.phi3_helper
      • experimental_experiment.torch_models.phi_helper
      • experimental_experiment.torch_models.training_helper
    • experimental_experiment.xbuilder
      • experimental_experiment.xbuilder._dtype_helper
      • experimental_experiment.xbuilder._graph_builder_runtime
      • experimental_experiment.xbuilder._onnx_helper
      • experimental_experiment.xbuilder._shape_helper
      • experimental_experiment.xbuilder.expression_dimension
      • experimental_experiment.xbuilder.graph_builder
      • experimental_experiment.xbuilder.graph_builder_opset
      • experimental_experiment.xbuilder.model_container
      • experimental_experiment.xbuilder.optimization_options
      • experimental_experiment.xbuilder.shape_type_compute
      • experimental_experiment.xbuilder.type_inference
    • experimental_experiment.xoptim
      • experimental_experiment.xoptim.patterns_investigation
        • experimental_experiment.xoptim.patterns_investigation.element_wise
        • experimental_experiment.xoptim.patterns_investigation.llm_patterns
      • experimental_experiment.xoptim.patterns_ml
        • experimental_experiment.xoptim.patterns_ml.tree_ensemble
      • experimental_experiment.xoptim.patterns_exp
        • experimental_experiment.xoptim.patterns_exp.binary_operators
        • experimental_experiment.xoptim.patterns_exp.constant_of_shape_scatter_nd
        • experimental_experiment.xoptim.patterns_exp.constants
        • experimental_experiment.xoptim.patterns_exp.simple_rotary
        • experimental_experiment.xoptim.patterns_exp.unary_operators
        • experimental_experiment.xoptim.patterns_exp.where_replace
      • experimental_experiment.xoptim.patterns
        • experimental_experiment.xoptim.patterns.onnx_any
        • experimental_experiment.xoptim.patterns.onnx_cast
        • experimental_experiment.xoptim.patterns.onnx_clip
        • experimental_experiment.xoptim.patterns.onnx_conv
        • experimental_experiment.xoptim.patterns.onnx_dropout
        • experimental_experiment.xoptim.patterns.onnx_equal
        • experimental_experiment.xoptim.patterns.onnx_expand
        • experimental_experiment.xoptim.patterns.onnx_functions
        • experimental_experiment.xoptim.patterns.onnx_layer_normalization
        • experimental_experiment.xoptim.patterns.onnx_matmul
        • experimental_experiment.xoptim.patterns.onnx_mul
        • experimental_experiment.xoptim.patterns.onnx_reduce
        • experimental_experiment.xoptim.patterns.onnx_reshape
        • experimental_experiment.xoptim.patterns.onnx_rotary
        • experimental_experiment.xoptim.patterns.onnx_slice
        • experimental_experiment.xoptim.patterns.onnx_split
        • experimental_experiment.xoptim.patterns.onnx_sub
        • experimental_experiment.xoptim.patterns.onnx_sequence
        • experimental_experiment.xoptim.patterns.onnx_transpose
        • experimental_experiment.xoptim.patterns.onnx_unsqueeze
      • experimental_experiment.xoptim.patterns_ort
        • experimental_experiment.xoptim.patterns_ort.activation
        • experimental_experiment.xoptim.patterns_ort.activation_grad
        • experimental_experiment.xoptim.patterns_ort.fused_conv
        • experimental_experiment.xoptim.patterns_ort.fused_matmul
        • experimental_experiment.xoptim.patterns_ort.gather_grad
        • experimental_experiment.xoptim.patterns_ort.simplified_layer_normalization
      • experimental_experiment.xoptim.patterns_fix
        • experimental_experiment.xoptim.patterns_fix.add_reduction_scatter_nd
      • experimental_experiment.xoptim.graph_builder_optim
      • experimental_experiment.xoptim.order_optim
      • experimental_experiment.xoptim.patterns_api
    • experimental_experiment.torch_dynamo
      • experimental_experiment.torch_dynamo._dynamo_exporter
      • experimental_experiment.torch_dynamo.backend_helper
      • experimental_experiment.torch_dynamo.debug_backend
      • experimental_experiment.torch_dynamo.fast_backend
      • experimental_experiment.torch_dynamo.partition
    • experimental_experiment.torch_bench
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner_agg
      • experimental_experiment.torch_bench._bash_bench_benchmark_runner_agg_helper
      • experimental_experiment.torch_bench._bash_bench_cmd
      • experimental_experiment.torch_bench._bash_bench_model_runner
      • experimental_experiment.torch_bench._bash_bench_models_helper
      • experimental_experiment.torch_bench._bash_bench_set_dummies
      • experimental_experiment.torch_bench._bash_bench_set_explicit
      • experimental_experiment.torch_bench._bash_bench_set_huggingface
      • experimental_experiment.torch_bench._bash_bench_set_huggingface_big
      • experimental_experiment.torch_bench._bash_bench_set_issues
      • experimental_experiment.torch_bench._bash_bench_set_timm
      • experimental_experiment.torch_bench._bash_bench_set_torchbench
      • experimental_experiment.torch_bench._bash_bench_set_torchbench_ado
      • experimental_experiment.torch_bench._bash_bench_suites
      • experimental_experiment.torch_bench._dort_cmd_common
      • experimental_experiment.torch_bench._dort_cmd_common_models
      • experimental_experiment.torch_bench.bash_bench_agg
      • experimental_experiment.torch_bench.bash_bench_explicit
      • experimental_experiment.torch_bench.bash_bench_huggingface
      • experimental_experiment.torch_bench.bash_bench_huggingface_big
      • experimental_experiment.torch_bench.bash_bench_issues
      • experimental_experiment.torch_bench.bash_bench_timm
      • experimental_experiment.torch_bench.bash_bench_torchbench
      • experimental_experiment.torch_bench.bash_bench_torchbench_ado
      • experimental_experiment.torch_bench.bash_bench_untrained
      • experimental_experiment.torch_bench.check_model
      • experimental_experiment.torch_bench.dort_bench
      • experimental_experiment.torch_bench.dort_bench_profile
      • experimental_experiment.torch_bench.dort_profile
      • experimental_experiment.torch_bench.export_model
      • experimental_experiment.torch_bench.export_model_helper
    • experimental_experiment._bench_test
    • experimental_experiment._command_lines_parser
    • experimental_experiment.args
    • experimental_experiment.bench_run
    • experimental_experiment.checks
    • experimental_experiment.ext_test_case
    • experimental_experiment.helpers
    • experimental_experiment.memory_peak
    • experimental_experiment.mini_onnx_builder
    • experimental_experiment.model_run
    • experimental_experiment.onnx_tools
    • experimental_experiment.torch_test_helper
  • Galleries of Examples and Recipes
    • Examples Gallery
      • 101: A custom backend for torch
      • 101: Linear Regression and export to ONNX
      • 101: Onnx Model Optimization based on Pattern Rewriting
      • 101: Onnx Model Rewriting
      • 101: Profile an existing model with onnxruntime
      • 101: Some dummy examples with torch.export.export
      • 102: Convolution and Matrix Multiplication
      • 102: Examples with onnxscript
      • 102: Fuse kernels in a small Llama Model
      • 102: Measure LLAMA speed
      • 102: Tweak onnx export
      • 201: Evaluate DORT
      • 201: Evaluate DORT Training
      • 201: Evaluate different ways to export a torch model to ONNX
      • 301: Compares LLAMA exporters
      • 301: Compares LLAMA exporters for onnxrt backend
    • Exporter Recipes Gallery
      • A few tricks about dynamic shapes
      • Do no use Module as inputs!
      • Export a model using a custom type as input
      • Linear Regression and export to ONNX
      • Measures the exporter success on many test cases
      • to_onnx and Phi-2
      • to_onnx and a custom operator inplace
      • to_onnx and a custom operator registered with a function
      • to_onnx and a model with a loop (scan)
      • to_onnx and a model with a test
      • to_onnx and submodules from LLMs
      • torch.onnx.export and Phi-2
      • torch.onnx.export and a model with a test
  • Command Lines
    • Benchmarks from the command line
      • experimental_experiment.torch_bench.dort_bench
      • experimental_experiment.torch_bench.dort_profile
      • Interesting scripts or command lines
      • Measuring the exporters on a short list of sets of models
    • Tools from the command line
      • python -m experimental_experiment lighten and unlighten
      • python -m experimental_experiment optimize
      • python -m experimental_experiment run
  • Miscellaneous
    • Export Times
    • Long Outputs uneasy to read
    • Supported Models By the Custom Backend
      • LLaMa
      • Mistral
      • Phi
      • Phi3

More

  • Change Logs
  • License
Back to top
View this page

experimental_experiment.torch_interpreter.patchesΒΆ

modules

  • experimental_experiment.torch_interpreter.patches.patch_torch
  • experimental_experiment.torch_interpreter.patches.patch_transformers
Next
experimental_experiment.torch_interpreter.patches.patch_torch
Previous
experimental_experiment.torch_interpreter.oxs_opset
Copyright © 2023-2024
Made with Sphinx and @pradyunsg's Furo