Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
onnx-diagnostic 0.7.16 documentation
Logo
onnx-diagnostic 0.7.16 documentation

Contents

  • Patches Explained
    • Exporter Status
      • Exported Programs with Dynamic Shapes
      • Exported ONNX with Dynamic Shapes
      • Coverage of the Patches
  • API of onnx_diagnostic
    • onnx_diagnostic.export
      • onnx_diagnostic.export.dynamic_shapes
      • onnx_diagnostic.export.shape_helper
      • onnx_diagnostic.export.validate
    • onnx_diagnostic.helpers
      • onnx_diagnostic.helpers.args_helper
      • onnx_diagnostic.helpers.bench_run
      • onnx_diagnostic.helpers.cache_helper
      • onnx_diagnostic.helpers.config_helper
      • onnx_diagnostic.helpers.doc_helper
      • onnx_diagnostic.helpers.fake_tensor_helper
      • onnx_diagnostic.helpers.graph_helper
      • onnx_diagnostic.helpers.helper
      • onnx_diagnostic.helpers._log_helper
      • onnx_diagnostic.helpers.log_helper
      • onnx_diagnostic.helpers.memory_peak
      • onnx_diagnostic.helpers.mini_onnx_builder
      • onnx_diagnostic.helpers.model_builder_helper
      • onnx_diagnostic.helpers.onnx_helper
      • onnx_diagnostic.helpers.ort_session
      • onnx_diagnostic.helpers.rt_helper
      • onnx_diagnostic.helpers.torch_helper
    • onnx_diagnostic.reference
      • onnx_diagnostic.reference.ops
        • onnx_diagnostic.reference.ops.op_add_add_mul_mul
        • onnx_diagnostic.reference.ops.op_average_pool_grad
        • onnx_diagnostic.reference.ops.op_cast_like
        • onnx_diagnostic.reference.ops.op_complex
        • onnx_diagnostic.reference.ops.op_concat
        • onnx_diagnostic.reference.ops.op_constant_of_shape
        • onnx_diagnostic.reference.ops.op_fused_matmul
        • onnx_diagnostic.reference.ops.op_gather_grad
        • onnx_diagnostic.reference.ops.op_memcpy_host
        • onnx_diagnostic.reference.ops.op_mul_sigmoid
        • onnx_diagnostic.reference.ops.op_negxplus1
        • onnx_diagnostic.reference.ops.op_quick_gelu
        • onnx_diagnostic.reference.ops.op_replace_zero
        • onnx_diagnostic.reference.ops.op_rotary
        • onnx_diagnostic.reference.ops.op_qlinear_average_pool
        • onnx_diagnostic.reference.ops.op_qlinear_conv
        • onnx_diagnostic.reference.ops.op_scatter_elements
        • onnx_diagnostic.reference.ops.op_scatternd_of_shape
        • onnx_diagnostic.reference.ops.op_simplified_layer_normalization
        • onnx_diagnostic.reference.ops.op_skip_layer_normalization
        • onnx_diagnostic.reference.ops.op_slice
        • onnx_diagnostic.reference.ops.op_transpose_cast
        • onnx_diagnostic.reference.ops.op_tri_matrix
      • onnx_diagnostic.reference.torch_ops
        • onnx_diagnostic.reference.torch_ops.access_ops
        • onnx_diagnostic.reference.torch_ops.binary_ops
        • onnx_diagnostic.reference.torch_ops.controlflow_ops
        • onnx_diagnostic.reference.torch_ops.generator_ops
        • onnx_diagnostic.reference.torch_ops.nn_ops
        • onnx_diagnostic.reference.torch_ops.other_ops
        • onnx_diagnostic.reference.torch_ops.reduce_ops
        • onnx_diagnostic.reference.torch_ops.sequence_ops
        • onnx_diagnostic.reference.torch_ops.shape_ops
        • onnx_diagnostic.reference.torch_ops.unary_ops
      • onnx_diagnostic.reference.evaluator
      • onnx_diagnostic.reference.quantized_tensor
      • onnx_diagnostic.reference.ort_evaluator
      • onnx_diagnostic.reference.report_results_comparison
      • onnx_diagnostic.reference.torch_evaluator
    • onnx_diagnostic.tasks
      • onnx_diagnostic.tasks.automatic_speech_recognition
      • onnx_diagnostic.tasks.fill_mask
      • onnx_diagnostic.tasks.feature_extraction
      • onnx_diagnostic.tasks.image_classification
      • onnx_diagnostic.export.image_text_to_text
      • onnx_diagnostic.tasks.mixture_of_expert
      • onnx_diagnostic.tasks.object_detection
      • onnx_diagnostic.tasks.sentence_similarity
      • onnx_diagnostic.tasks.summarization
      • onnx_diagnostic.tasks.text_classification
      • onnx_diagnostic.tasks.text_generation
      • onnx_diagnostic.tasks.text_to_image
      • onnx_diagnostic.tasks.text2text_generation
      • onnx_diagnostic.tasks.zero_shot_image_classification
    • onnx_diagnostic.torch_export_patches
      • onnx_diagnostic.torch_export_patches.eval
        • onnx_diagnostic.torch_export_patches.eval.model_cases
      • onnx_diagnostic.torch_export_patches.onnx_export_errors
      • onnx_diagnostic.torch_export_patches.onnx_export_serialization
      • onnx_diagnostic.torch_export_patches.patches
        • onnx_diagnostic.torch_export_patches.patches.patch_torch
        • onnx_diagnostic.torch_export_patches.patches.patch_transformers
      • onnx_diagnostic.torch_export_patches.patch_expressions
      • onnx_diagnostic.torch_export_patches.patch_inputs
      • onnx_diagnostic.torch_export_patches.patch_module
      • onnx_diagnostic.torch_export_patches.patch_module_helper
      • onnx_diagnostic.torch_export_patches.serialization
        • onnx_diagnostic.torch_export_patches.serialization.diffusers_impl
        • onnx_diagnostic.torch_export_patches.serialization.transformers_impl
    • onnx_diagnostic.torch_models
      • onnx_diagnostic.torch_models.code_sample
      • onnx_diagnostic.torch_models.hghub
        • onnx_diagnostic.torch_models.hghub.hub_api
        • onnx_diagnostic.torch_models.hghub.hub_data
        • onnx_diagnostic.torch_models.hghub.model_inputs
      • onnx_diagnostic.torch_models.llms
      • onnx_diagnostic.torch_models.validate
    • onnx_diagnostic.torch_onnx
      • onnx_diagnostic.torch_onnx.runtime_info
      • onnx_diagnostic.torch_onnx.sbs
    • onnx_diagnostic.api
    • onnx_diagnostic.ext_test_case
  • Command Lines
    • -m onnx_diagnostic config … prints the config for a model id
    • -m onnx_diagnostic validate … validate a model id
  • Examples Gallery
    • Dumps intermediate results of a torch model
    • Dynamic Shapes for *args, **kwargs
    • Export Tiny-LLM with patches
    • Export microsoft/phi-2
    • Export with DynamicCache and guessed dynamic shapes
    • Export with dynamic dimensions in {0,1}
    • Export with dynamic dimensions in {0,1} into ONNX
    • Export with dynamic dimensions in {0,1} into ONNX (custom)
    • Find and fix an export issue due to dynamic shapes
    • Find where a model is failing by running submodels
    • Intermediate results with (ONNX) ReferenceEvaluator
    • Intermediate results with onnxruntime
    • Steel method forward to guess inputs and dynamic shapes (with Tiny-LLM)
    • Test the export on untrained models
  • Common Export Issues
    • 0, 1, 2 for a Dynamic Dimension in the dummy example to export a model
    • Builds dynamic shapes from any input
    • Cannot export torch.sym_max(x.shape[0], y.shape[0])
    • Do not use python int with dynamic shapes
    • Export a model with a control flow (If)
    • Half certain nonzero
    • JSON returns list when the original dynamic shapes are list or tuple
    • Use DYNAMIC or AUTO when exporting if dynamic shapes has constraints
  • Technical Details
    • Dynamic Shapes and Broadcasting
    • LayerNormalization implementation cannot be exchanged
    • Reproducible Parallelized Reduction is difficult

More

  • Change Logs
  • License
Back to top
View this page

Technical Details¶

Dynamic Shapes and Broadcasting

Dynamic Shapes and Broadcasting

LayerNormalization implementation cannot be exchanged

LayerNormalization implementation cannot be exchanged

Reproducible Parallelized Reduction is difficult

Reproducible Parallelized Reduction is difficult

Download all examples in Python source code: auto_technical_python.zip

Download all examples in Jupyter notebooks: auto_technical_jupyter.zip

Gallery generated by Sphinx-Gallery

Next
Dynamic Shapes and Broadcasting
Previous
Use DYNAMIC or AUTO when exporting if dynamic shapes has constraints
Copyright © 2025
Made with Sphinx and @pradyunsg's Furo