:orphan:

PyTorch Gallery
===============

Examples about exporting PyTorch models to ONNX, model patching, and input observation.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Before exporting a PyTorch model with torch.export.export, a set of patches must be applied to work around limitations in the PyTorch exporter. This example shows how to:">

.. only:: html

  .. image:: /auto_examples_torch/images/thumb/sphx_glr_plot_patch_model_thumb.png
    :alt:

  :doc:`/auto_examples_torch/plot_patch_model`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Applying patches to a model and displaying the diff</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The main issue when exporting a LLM is the example on HuggingFace is based on method generate but we only need to export the forward method. Example l-plot-input-observer-transformers gives details on how to guess dummy inputs and dynamic shapes to do so. Let&#x27;s see how to simplify that.">

.. only:: html

  .. image:: /auto_examples_torch/images/thumb/sphx_glr_plot_input_observer_tiny_llm_thumb.png
    :alt:

  :doc:`/auto_examples_torch/plot_input_observer_tiny_llm`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Export a LLM with InputObserver (with Tiny-LLM)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="yobx.torch.input_observer.InputObserver is a context manager that steals a model&#x27;s forward method during inference to record every set of inputs and outputs.  After the context exits, the collected data can be used to:">

.. only:: html

  .. image:: /auto_examples_torch/images/thumb/sphx_glr_plot_input_observer_transformers_thumb.png
    :alt:

  :doc:`/auto_examples_torch/plot_input_observer_transformers`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">InputObserver with Transformers Cache</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="yobx.torch.input_observer.InputObserver is a context manager that steals a model&#x27;s forward method during inference to record every set of inputs and outputs.  After the context exits, the collected data can be used to:">

.. only:: html

  .. image:: /auto_examples_torch/images/thumb/sphx_glr_plot_input_observer_thumb.png
    :alt:

  :doc:`/auto_examples_torch/plot_input_observer`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">InputObserver: recording inputs for ONNX export</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="torch.export.export requires every object that appears as a model input or output to be decomposable into a flat list of torch.Tensor objects.  torch.utils._pytree handles this decomposition, but it only knows about built-in Python containers. Custom classes — including all the cache types from transformers — must be explicitly registered before exporting.">

.. only:: html

  .. image:: /auto_examples_torch/images/thumb/sphx_glr_plot_flattening_thumb.png
    :alt:

  :doc:`/auto_examples_torch/plot_flattening`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Registering a custom class as a pytree node</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples_torch/plot_patch_model
   /auto_examples_torch/plot_input_observer_tiny_llm
   /auto_examples_torch/plot_input_observer_transformers
   /auto_examples_torch/plot_input_observer
   /auto_examples_torch/plot_flattening


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: auto_examples_torch_python.zip </auto_examples_torch/auto_examples_torch_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: auto_examples_torch_jupyter.zip </auto_examples_torch/auto_examples_torch_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_