Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
onnx-diagnostic 0.8.3 documentation
Logo
onnx-diagnostic 0.8.3 documentation

Contents

  • Patches Explained
    • Exporter Status
      • Exported Programs with Dynamic Shapes
      • Exported ONNX with Dynamic Shapes
      • Coverage of the Patches
      • Patches Diff
  • API of onnx_diagnostic
    • onnx_diagnostic.export
      • onnx_diagnostic.export.api
      • onnx_diagnostic.export.control_flow_onnx
      • onnx_diagnostic.export.dynamic_shapes
      • onnx_diagnostic.export.onnx_plug
      • onnx_diagnostic.export.shape_helper
      • onnx_diagnostic.export.validate
    • onnx_diagnostic.helpers
      • onnx_diagnostic.helpers.args_helper
      • onnx_diagnostic.helpers.bench_run
      • onnx_diagnostic.helpers.cache_helper
      • onnx_diagnostic.helpers.config_helper
      • onnx_diagnostic.helpers.doc_helper
      • onnx_diagnostic.helpers.fake_tensor_helper
      • onnx_diagnostic.helpers.graph_helper
      • onnx_diagnostic.helpers.helper
      • onnx_diagnostic.helpers._log_helper
      • onnx_diagnostic.helpers.log_helper
      • onnx_diagnostic.helpers.memory_peak
      • onnx_diagnostic.helpers.mini_onnx_builder
      • onnx_diagnostic.helpers.model_builder_helper
      • onnx_diagnostic.helpers.onnx_helper
      • onnx_diagnostic.helpers.ort_session
      • onnx_diagnostic.helpers.rt_helper
      • onnx_diagnostic.helpers.torch_fx_graph_helper
      • onnx_diagnostic.helpers.torch_helper
    • onnx_diagnostic.reference
      • onnx_diagnostic.reference.ops
        • onnx_diagnostic.reference.ops.op_add_add_mul_mul
        • onnx_diagnostic.reference.ops.op_average_pool_grad
        • onnx_diagnostic.reference.ops.op_cast_like
        • onnx_diagnostic.reference.ops.op_complex
        • onnx_diagnostic.reference.ops.op_concat
        • onnx_diagnostic.reference.ops.op_constant_of_shape
        • onnx_diagnostic.reference.ops.op_fused_matmul
        • onnx_diagnostic.reference.ops.op_gather_grad
        • onnx_diagnostic.reference.ops.op_memcpy_host
        • onnx_diagnostic.reference.ops.op_mul_sigmoid
        • onnx_diagnostic.reference.ops.op_negxplus1
        • onnx_diagnostic.reference.ops.op_quick_gelu
        • onnx_diagnostic.reference.ops.op_replace_zero
        • onnx_diagnostic.reference.ops.op_rotary
        • onnx_diagnostic.reference.ops.op_qlinear_average_pool
        • onnx_diagnostic.reference.ops.op_qlinear_conv
        • onnx_diagnostic.reference.ops.op_scatter_elements
        • onnx_diagnostic.reference.ops.op_scatternd_of_shape
        • onnx_diagnostic.reference.ops.op_simplified_layer_normalization
        • onnx_diagnostic.reference.ops.op_skip_layer_normalization
        • onnx_diagnostic.reference.ops.op_slice
        • onnx_diagnostic.reference.ops.op_transpose_cast
        • onnx_diagnostic.reference.ops.op_tri_matrix
      • onnx_diagnostic.reference.torch_ops
        • onnx_diagnostic.reference.torch_ops.access_ops
        • onnx_diagnostic.reference.torch_ops.binary_ops
        • onnx_diagnostic.reference.torch_ops.controlflow_ops
        • onnx_diagnostic.reference.torch_ops.generator_ops
        • onnx_diagnostic.reference.torch_ops.nn_ops
        • onnx_diagnostic.reference.torch_ops.other_ops
        • onnx_diagnostic.reference.torch_ops.reduce_ops
        • onnx_diagnostic.reference.torch_ops.sequence_ops
        • onnx_diagnostic.reference.torch_ops.shape_ops
        • onnx_diagnostic.reference.torch_ops.unary_ops
      • onnx_diagnostic.reference.evaluator
      • onnx_diagnostic.reference.quantized_tensor
      • onnx_diagnostic.reference.ort_evaluator
      • onnx_diagnostic.reference.report_results_comparison
      • onnx_diagnostic.reference.torch_evaluator
    • onnx_diagnostic.tasks
      • onnx_diagnostic.tasks.automatic_speech_recognition
      • onnx_diagnostic.tasks.fill_mask
      • onnx_diagnostic.tasks.feature_extraction
      • onnx_diagnostic.tasks.image_classification
      • onnx_diagnostic.export.image_text_to_text
      • onnx_diagnostic.tasks.mixture_of_expert
      • onnx_diagnostic.tasks.object_detection
      • onnx_diagnostic.tasks.sentence_similarity
      • onnx_diagnostic.tasks.summarization
      • onnx_diagnostic.tasks.text_classification
      • onnx_diagnostic.tasks.text_generation
      • onnx_diagnostic.tasks.text_to_image
      • onnx_diagnostic.tasks.text2text_generation
      • onnx_diagnostic.tasks.zero_shot_image_classification
    • onnx_diagnostic.torch_export_patches
      • onnx_diagnostic.torch_export_patches.eval
        • onnx_diagnostic.torch_export_patches.eval.model_cases
      • onnx_diagnostic.torch_export_patches.onnx_export_errors
      • onnx_diagnostic.torch_export_patches.onnx_export_serialization
      • onnx_diagnostic.torch_export_patches.patches
        • onnx_diagnostic.torch_export_patches.patches.patch_torch
        • onnx_diagnostic.torch_export_patches.patches.patch_transformers
      • onnx_diagnostic.torch_export_patches.patch_details
      • onnx_diagnostic.torch_export_patches.patch_expressions
      • onnx_diagnostic.torch_export_patches.patch_inputs
      • onnx_diagnostic.torch_export_patches.patch_module
      • onnx_diagnostic.torch_export_patches.patch_module_helper
      • onnx_diagnostic.torch_export_patches.serialization
        • onnx_diagnostic.torch_export_patches.serialization.diffusers_impl
        • onnx_diagnostic.torch_export_patches.serialization.transformers_impl
    • onnx_diagnostic.torch_models
      • onnx_diagnostic.torch_models.code_sample
      • onnx_diagnostic.torch_models.hghub
        • onnx_diagnostic.torch_models.hghub.hub_api
        • onnx_diagnostic.torch_models.hghub.hub_data
        • onnx_diagnostic.torch_models.hghub.model_inputs
      • onnx_diagnostic.torch_models.llms
      • onnx_diagnostic.torch_models.validate
    • onnx_diagnostic.torch_onnx
      • onnx_diagnostic.torch_onnx.runtime_info
      • onnx_diagnostic.torch_onnx.sbs
      • onnx_diagnostic.torch_onnx.sbs_dataclasses
    • onnx_diagnostic.api
    • onnx_diagnostic.ext_test_case
  • Command Lines
    • -m onnx_diagnostic config … prints the config for a model id
    • -m onnx_diagnostic sbs … runs a side-by-side torch/onnx
    • -m onnx_diagnostic validate … validate a model id
  • Examples Gallery
    • Dumps intermediate results of a torch model
    • Dynamic Shapes for *args, **kwargs
    • Export Tiny-LLM with patches
    • Export microsoft/phi-2
    • Export with DynamicCache and guessed dynamic shapes
    • Export with dynamic dimensions in {0,1}
    • Export with dynamic dimensions in {0,1} into ONNX
    • Export with dynamic dimensions in {0,1} into ONNX (custom)
    • Find and fix an export issue due to dynamic shapes
    • Find where a model is failing by running submodels
    • Intermediate results with (ONNX) ReferenceEvaluator
    • Intermediate results with onnxruntime
    • Steel method forward to guess inputs and dynamic shapes (with Tiny-LLM)
    • Test the export on untrained models
  • Common Export Issues
    • 0, 1, 2 for a Dynamic Dimension in the dummy example to export a model
    • Builds dynamic shapes from any input
    • Cannot export torch.sym_max(x.shape[0], y.shape[0])
    • Do not use python int with dynamic shapes
    • Export a model with a control flow (If)
    • Half certain nonzero
    • JSON returns list when the original dynamic shapes are list or tuple
    • Use DYNAMIC or AUTO when exporting if dynamic shapes has constraints
  • Technical Details
    • Dynamic Shapes and Broadcasting
    • From a LLM to processing a prompt
    • Gemm or Matmul + Add
    • LayerNormalization implementation cannot be exchanged
    • Reproducible Parallelized Reduction is difficult

More

  • Change Logs
  • License
Back to top
View this page

onnx_diagnostic.torch_export_patches.patches.patch_transformers¶

onnx_diagnostic.torch_export_patches.patches.patch_transformers.get_transformers_plugs() → List[EagerDirectReplacementWithOnnx][source][source]¶

Returns the necessary plugs to rewrite models.

Next
onnx_diagnostic.torch_export_patches.patch_details
Previous
onnx_diagnostic.torch_export_patches.patches.patch_torch
Copyright © 2025
Made with Sphinx and @pradyunsg's Furo
On this page
  • onnx_diagnostic.torch_export_patches.patches.patch_transformers
    • get_transformers_plugs()