experimental_experiment.ext_test_case

The module contains the main class ExtTestCase which adds specific functionalities to this project.

class experimental_experiment.ext_test_case.ExtTestCase(methodName='runTest')

Inherits from unittest.TestCase and adds specific comprison functions and other helpers.

assertAlmostEqual(expected: ndarray, value: ndarray, atol: float = 0, rtol: float = 0)

In the name

assertEmpty(value: Any)

In the name

assertEqual(expected: Any, value: Any, msg: str = '')

Overwrites the error message to get a more explicit message about what is what.

assertEqualArray(expected: ndarray, value: ndarray, atol: float = 0, rtol: float = 0, msg: str | None = None)

In the name

assertEqualArrays(expected: Sequence[ndarray], value: Sequence[ndarray], atol: float = 0, rtol: float = 0, msg: str | None = None)

In the name

assertExists(name)

Checks the existing of a file.

assertGreaterOrEqual(a, b, msg=None)

In the name

assertIn(tofind: str, text: str)

Just like self.assertTrue(a in b), but with a nicer default message.

assertNotEmpty(value: Any)

In the name

assertRaise(fct: Callable, exc_type: type[Exception])

In the name

assertStartsWith(prefix: str, full: str)

In the name

capture(fct: Callable)

Runs a function and capture standard output and error.

Parameters:

fct – function to run

Returns:

result of fct, output, error

dump_onnx(name: str, proto: Any, folder: str | None = None) → str

Dumps an onnx file.

get_dump_file(name: str, folder: str | None = None) → str

Returns a filename to dump a model.

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

classmethod tearDownClass()

Hook method for deconstructing the class fixture after running all tests in the class.

tryCall(fct: Callable, msg: str | None = None, none_if: str | None = None) → Any | None

Calls the function, catch any error.

Parameters:

• fct – function to call

• msg – error message to display if failing

• none_if – returns None if this substring is found in the error message

Returns:

output of fct

experimental_experiment.ext_test_case.dump_dort_onnx(fn)

Context manager to dump onnx model created by dort.

experimental_experiment.ext_test_case.get_figure(ax)

Returns the figure of a matplotlib figure.

experimental_experiment.ext_test_case.has_cuda() → bool

Returns torch.cuda.is_available().

experimental_experiment.ext_test_case.has_executorch(version: str = '', msg: str = '') → Callable

Tells if ExecuTorch is installed.

experimental_experiment.ext_test_case.has_onnxruntime_training(push_back_batch: bool = False)

Tells if onnxruntime_training is installed.

experimental_experiment.ext_test_case.hide_stdout(f: Callable | None = None) → Callable

Catches warnings.

Parameters:

f – the function is called with the stdout as an argument

experimental_experiment.ext_test_case.ignore_warnings(warns: List[Warning]) → Callable

Catches warnings.

Parameters:

warns – warnings to ignore

experimental_experiment.ext_test_case.is_azure() → bool

Tells if the job is running on Azure DevOps.

experimental_experiment.ext_test_case.long_test(msg: str = '') → Callable

Catches warnings.

Parameters:

f – the function is called with the stdout as an argument

experimental_experiment.ext_test_case.measure_time(stmt: str | Callable, context: Dict[str, Any] | None = None, repeat: int = 10, number: int = 50, warmup: int = 1, div_by_number: bool = True, max_time: float | None = None) → Dict[str, str | int | float]

Measures a statement and returns the results as a dictionary.

Parameters:

• stmt – string or callable

• context – variable to know in a dictionary

• repeat – average over repeat experiment

• number – number of executions in one row

• warmup – number of iteration to do before starting the real measurement

• div_by_number – divide by the number of executions

• max_time – execute the statement until the total goes beyond this time (approximatively), repeat is ignored, div_by_number must be set to True

Returns:

dictionary

<<<

from pprint import pprint
from math import cos
from experimental_experiment.ext_test_case import measure_time

res = measure_time(lambda: cos(0.5))
pprint(res)

>>>

    {'average': np.float64(7.142600952647625e-08),
     'context_size': 64,
     'deviation': np.float64(5.676754166519193e-09),
     'max_exec': np.float64(8.579998393543065e-08),
     'min_exec': np.float64(6.724003469571471e-08),
     'number': 50,
     'repeat': 10,
     'ttime': np.float64(7.142600952647626e-07),
     'warmup_time': 1.513199822511524e-05}

See Timer.repeat for a better understanding of parameter repeat and number. The function returns a duration corresponding to number times the execution of the main statement.

experimental_experiment.ext_test_case.require_diffusers(version: str, msg: str = '', or_older_than: str | None = None) → Callable

Skips a unit test if transformers is not recent enough.

experimental_experiment.ext_test_case.requires_cuda(msg: str = '', version: str = '', memory: int = 0)

Skips a test if cuda is not available.

Parameters:

• msg – to overwrite the message

• version – minimum version

• memory – minimun number of Gb to run the test

experimental_experiment.ext_test_case.requires_executorch(version: str, msg: str = '') → Callable

Skips a unit test if executorch is not recent enough.

experimental_experiment.ext_test_case.requires_monai(version: str = '', msg: str = '') → Callable

Skips a unit test if monai is not recent enough.

experimental_experiment.ext_test_case.requires_numpy(version: str, msg: str = '') → Callable

Skips a unit test if numpy is not recent enough.

experimental_experiment.ext_test_case.requires_onnx(version: str, msg: str = '') → Callable

Skips a unit test if onnx is not recent enough.

experimental_experiment.ext_test_case.requires_onnxruntime(version: str, msg: str = '') → Callable

Skips a unit test if onnxruntime is not recent enough.

experimental_experiment.ext_test_case.requires_onnxruntime_training(push_back_batch: bool = False, msg: str = '') → Callable

Skips a unit test if onnxruntime is not onnxruntime_training.

experimental_experiment.ext_test_case.requires_onnxscript(version: str, msg: str = '') → Callable

Skips a unit test if onnxscript is not recent enough.

experimental_experiment.ext_test_case.requires_pyinstrument(version: str = '', msg: str = '') → Callable

Skips a unit test if pyinstrument is not recent enough.

experimental_experiment.ext_test_case.requires_sklearn(version: str, msg: str = '') → Callable

Skips a unit test if scikit-learn is not recent enough.

experimental_experiment.ext_test_case.requires_torch(version: str, msg: str = '') → Callable

Skips a unit test if pytorch is not recent enough.

experimental_experiment.ext_test_case.requires_transformers(version: str, msg: str = '', or_older_than: str | None = None) → Callable

Skips a unit test if transformers is not recent enough.

experimental_experiment.ext_test_case.requires_vocos(version: str = '', msg: str = '') → Callable

Skips a unit test if vocos is not recent enough.

experimental_experiment.ext_test_case.requires_zoo(msg: str = '') → Callable

Skips a unit test if environment variable ZOO is not equal to 1.

experimental_experiment.ext_test_case.skipif_ci_apple(msg) → Callable

Skips a unit test if it runs on azure pipeline on Windows.

experimental_experiment.ext_test_case.skipif_ci_linux(msg) → Callable

Skips a unit test if it runs on azure pipeline on Linux.

experimental_experiment.ext_test_case.skipif_ci_windows(msg) → Callable

Skips a unit test if it runs on azure pipeline on Windows.

experimental_experiment.ext_test_case.skipif_not_onnxrt(msg) → Callable

Skips a unit test if it runs on azure pipeline on Windows.

experimental_experiment.ext_test_case.skipif_transformers(version_to_skip: str | Set[str], msg: str) → Callable

Skips a unit test if transformers has a specific version.

experimental_experiment.ext_test_case.statistics_on_file(filename: str) → Dict[str, str | int | float]

Computes statistics on a file.

<<<

import pprint
from experimental_experiment.ext_test_case import statistics_on_file, __file__

pprint.pprint(statistics_on_file(__file__))

>>>

    {'chars': 21980, 'ext': '.py', 'lines': 750}

experimental_experiment.ext_test_case.statistics_on_folder(folder: str | List[str], pattern: str = '.*[.]((py|rst))$', aggregation: int = 0) → List[Dict[str, str | int | float]]

Computes statistics on files in a folder.

Parameters:

• folder – folder or folders to investigate

• pattern – file pattern

• aggregation – show the first subfolders

Returns:

list of dictionaries

<<<

import os
import pprint
from experimental_experiment.ext_test_case import statistics_on_folder, __file__

pprint.pprint(statistics_on_folder(os.path.dirname(__file__)))

>>>

    [{'chars': 66, 'ext': '.py', 'lines': 3, 'name': '_bench_test.py'},
     {'chars': 6190, 'ext': '.py', 'lines': 193, 'name': 'torch_test_helper.py'},
     {'chars': 1807, 'ext': '.py', 'lines': 51, 'name': 'checks.py'},
     {'chars': 8416,
      'ext': '.py',
      'lines': 335,
      'name': '_command_lines_parser.py'},
     {'chars': 21980, 'ext': '.py', 'lines': 750, 'name': 'ext_test_case.py'},
     {'chars': 2725, 'ext': '.py', 'lines': 102, 'name': 'onnx_tools.py'},
     {'chars': 18176, 'ext': '.py', 'lines': 672, 'name': 'bench_run.py'},
     {'chars': 2876, 'ext': '.py', 'lines': 115, 'name': 'args.py'},
     {'chars': 4593, 'ext': '.py', 'lines': 157, 'name': 'helpers.py'},
     {'chars': 59, 'ext': '.py', 'lines': 3, 'name': '__init__.py'},
     {'chars': 5300, 'ext': '.py', 'lines': 158, 'name': 'model_run.py'},
     {'chars': 65, 'ext': '.py', 'lines': 3, 'name': '__main__.py'},
     {'chars': 4329, 'ext': '.py', 'lines': 187, 'name': 'memory_peak.py'},
     {'chars': 12052,
      'ext': '.py',
      'lines': 419,
      'name': 'gradient/grad_helper.py'},
     {'chars': 17619,
      'ext': '.py',
      'lines': 496,
      'name': 'gradient/loss_helper.py'},
     {'chars': 0, 'ext': '.py', 'lines': 0, 'name': 'gradient/__init__.py'},
     {'chars': 89, 'ext': '.py', 'lines': 2, 'name': 'gradient/ops/__init__.py'},
     {'chars': 671,
      'ext': '.py',
      'lines': 42,
      'name': 'gradient/ops/op_broadcast_gradient_args.py'},
     {'chars': 14210,
      'ext': '.py',
      'lines': 479,
      'name': 'reference/ort_evaluator.py'},
     {'chars': 5446, 'ext': '.py', 'lines': 196, 'name': 'reference/evaluator.py'},
     {'chars': 1111, 'ext': '.py', 'lines': 39, 'name': 'reference/__init__.py'},
     {'chars': 647,
      'ext': '.py',
      'lines': 27,
      'name': 'reference/ops/op_fused_matmul.py'},
     {'chars': 380,
      'ext': '.py',
      'lines': 14,
      'name': 'reference/ops/op_tri_matrix.py'},
     {'chars': 458,
      'ext': '.py',
      'lines': 17,
      'name': 'reference/ops/op_quick_gelu.py'},
     {'chars': 531, 'ext': '.py', 'lines': 15, 'name': 'reference/ops/op_slice.py'},
     {'chars': 907,
      'ext': '.py',
      'lines': 48,
      'name': 'reference/ops/op_constant_of_shape.py'},
     {'chars': 667,
      'ext': '.py',
      'lines': 16,
      'name': 'reference/ops/op_scatternd_of_shape.py'},
     {'chars': 147,
      'ext': '.py',
      'lines': 5,
      'name': 'reference/ops/op_negxplus1.py'},
     {'chars': 220,
      'ext': '.py',
      'lines': 6,
      'name': 'reference/ops/op_simplified_layer_normalization.py'},
     {'chars': 295,
      'ext': '.py',
      'lines': 10,
      'name': 'reference/ops/op_transpose_cast.py'},
     {'chars': 140,
      'ext': '.py',
      'lines': 7,
      'name': 'reference/ops/op_memcpy_host.py'},
     {'chars': 1405,
      'ext': '.py',
      'lines': 51,
      'name': 'reference/ops/op_average_pool_grad.py'},
     {'chars': 317,
      'ext': '.py',
      'lines': 9,
      'name': 'reference/ops/op_gather_grad.py'},
     {'chars': 224,
      'ext': '.py',
      'lines': 10,
      'name': 'reference/ops/op_replace_zero.py'},
     {'chars': 455,
      'ext': '.py',
      'lines': 17,
      'name': 'reference/ops/op_mul_sigmoid.py'},
     {'chars': 0, 'ext': '.py', 'lines': 0, 'name': 'reference/ops/__init__.py'},
     {'chars': 434,
      'ext': '.py',
      'lines': 16,
      'name': 'reference/ops/op_rotary.py'},
     {'chars': 383,
      'ext': '.py',
      'lines': 11,
      'name': 'reference/ops/op_concat.py'},
     {'chars': 1091,
      'ext': '.py',
      'lines': 44,
      'name': 'reference/ops/op_add_add_mul_mul.py'},
     {'chars': 2148,
      'ext': '.py',
      'lines': 79,
      'name': 'reference/ops/op_scatter_elements.py'},
     {'chars': 990,
      'ext': '.py',
      'lines': 31,
      'name': 'reference/ops/op_cast_like.py'},
     {'chars': 852, 'ext': '.py', 'lines': 23, 'name': 'convert/ort_helper.py'},
     {'chars': 4887,
      'ext': '.py',
      'lines': 149,
      'name': 'convert/convert_helper.py'},
     {'chars': 0, 'ext': '.py', 'lines': 0, 'name': 'convert/__init__.py'},
     {'chars': 6285, 'ext': '.py', 'lines': 377, 'name': 'plotting/data.py'},
     {'chars': 1893, 'ext': '.py', 'lines': 54, 'name': 'plotting/memory.py'},
     {'chars': 0, 'ext': '.py', 'lines': 0, 'name': 'plotting/__init__.py'},
     {'chars': 152, 'ext': '.py', 'lines': 4, 'name': 'torch_interpreter/_doc_.py'},
     {'chars': 8507,
      'ext': '.py',
      'lines': 322,
      'name': 'torch_interpreter/_aten_methods.py'},
     {'chars': 6826,
      'ext': '.py',
      'lines': 245,
      'name': 'torch_interpreter/dispatcher.py'},
     {'chars': 528,
      'ext': '.py',
      'lines': 21,
      'name': 'torch_interpreter/aten_methods.py'},
     {'chars': 44891,
      'ext': '.py',
      'lines': 1429,
      'name': 'torch_interpreter/interpreter.py'},
     {'chars': 24135,
      'ext': '.py',
      'lines': 745,
      'name': 'torch_interpreter/onnx_export.py'},
     {'chars': 793,
      'ext': '.py',
      'lines': 29,
      'name': 'torch_interpreter/_torch_helper.py'},
     {'chars': 87,
      'ext': '.py',
      'lines': 2,
      'name': 'torch_interpreter/_exceptions.py'},
     {'chars': 10212,
      'ext': '.py',
      'lines': 454,
      'name': 'torch_interpreter/_prims_functions.py'},
     {'chars': 179800,
      'ext': '.py',
      'lines': 7098,
      'name': 'torch_interpreter/_aten_functions.py'},
     {'chars': 8923,
      'ext': '.py',
      'lines': 293,
      'name': 'torch_interpreter/_aten_functions_attention.py'},
     {'chars': 1807,
      'ext': '.py',
      'lines': 59,
      'name': 'torch_interpreter/aten_functions.py'},
     {'chars': 382,
      'ext': '.py',
      'lines': 9,
      'name': 'torch_interpreter/__init__.py'},
     {'chars': 5675,
      'ext': '.py',
      'lines': 200,
      'name': 'torch_interpreter/oxs_dispatcher.py'},
     {'chars': 3927,
      'ext': '.py',
      'lines': 178,
      'name': 'torch_interpreter/oxs_opset.py'},
     {'chars': 3664,
      'ext': '.py',
      'lines': 120,
      'name': 'torch_interpreter/onnx_export_errors.py'},
     {'chars': 9132,
      'ext': '.py',
      'lines': 279,
      'name': 'torch_interpreter/export_options.py'},
     {'chars': 3970,
      'ext': '.py',
      'lines': 140,
      'name': 'torch_models/phi3_helper.py'},
     {'chars': 1651,
      'ext': '.py',
      'lines': 61,
      'name': 'torch_models/diffusion_model_helper.py'},
     {'chars': 5875,
      'ext': '.py',
      'lines': 204,
      'name': 'torch_models/dump_helper.py'},
     {'chars': 7525,
      'ext': '.py',
      'lines': 220,
      'name': 'torch_models/training_helper.py'},
     {'chars': 3414,
      'ext': '.py',
      'lines': 118,
      'name': 'torch_models/mistral_helper.py'},
     {'chars': 588, 'ext': '.py', 'lines': 19, 'name': 'torch_models/__init__.py'},
     {'chars': 14320,
      'ext': '.py',
      'lines': 575,
      'name': 'torch_models/llm_model_helper.py'},
     {'chars': 6031,
      'ext': '.py',
      'lines': 165,
      'name': 'torch_models/llama_helper.py'},
     {'chars': 3802,
      'ext': '.py',
      'lines': 132,
      'name': 'torch_models/phi_helper.py'},
     {'chars': 4090,
      'ext': '.py',
      'lines': 144,
      'name': 'xbuilder/type_inference.py'},
     {'chars': 4788,
      'ext': '.py',
      'lines': 192,
      'name': 'xbuilder/graph_builder_opset.py'},
     {'chars': 25453,
      'ext': '.py',
      'lines': 827,
      'name': 'xbuilder/shape_type_compute.py'},
     {'chars': 6997,
      'ext': '.py',
      'lines': 231,
      'name': 'xbuilder/model_container.py'},
     {'chars': 1716,
      'ext': '.py',
      'lines': 63,
      'name': 'xbuilder/expression_dimension.py'},
     {'chars': 174319,
      'ext': '.py',
      'lines': 5749,
      'name': 'xbuilder/graph_builder.py'},
     {'chars': 3266,
      'ext': '.py',
      'lines': 106,
      'name': 'xbuilder/_shape_helper.py'},
     {'chars': 2143,
      'ext': '.py',
      'lines': 80,
      'name': 'xbuilder/_dtype_helper.py'},
     {'chars': 6347,
      'ext': '.py',
      'lines': 259,
      'name': 'xbuilder/_onnx_helper.py'},
     {'chars': 339, 'ext': '.py', 'lines': 10, 'name': 'xbuilder/__init__.py'},
     {'chars': 11402,
      'ext': '.py',
      'lines': 399,
      'name': 'xbuilder/_graph_builder_runtime.py'},
     {'chars': 4045,
      'ext': '.py',
      'lines': 108,
      'name': 'xbuilder/optimization_options.py'},
     {'chars': 28881, 'ext': '.py', 'lines': 962, 'name': 'xoptim/patterns_api.py'},
     {'chars': 4219, 'ext': '.py', 'lines': 170, 'name': 'xoptim/order_optim.py'},
     {'chars': 2873, 'ext': '.py', 'lines': 105, 'name': 'xoptim/__init__.py'},
     {'chars': 30042,
      'ext': '.py',
      'lines': 1047,
      'name': 'xoptim/graph_builder_optim.py'},
     {'chars': 459,
      'ext': '.py',
      'lines': 15,
      'name': 'xoptim/patterns_investigation/__init__.py'},
     {'chars': 1213,
      'ext': '.py',
      'lines': 44,
      'name': 'xoptim/patterns_investigation/element_wise.py'},
     {'chars': 9813,
      'ext': '.py',
      'lines': 319,
      'name': 'xoptim/patterns_ml/tree_ensemble.py'},
     {'chars': 522,
      'ext': '.py',
      'lines': 18,
      'name': 'xoptim/patterns_ml/__init__.py'},
     {'chars': 3969,
      'ext': '.py',
      'lines': 122,
      'name': 'xoptim/patterns_exp/constant_of_shape_scatter_nd.py'},
     {'chars': 3148,
      'ext': '.py',
      'lines': 93,
      'name': 'xoptim/patterns_exp/constants.py'},
     {'chars': 2528,
      'ext': '.py',
      'lines': 76,
      'name': 'xoptim/patterns_exp/unary_operators.py'},
     {'chars': 2470,
      'ext': '.py',
      'lines': 81,
      'name': 'xoptim/patterns_exp/simple_rotary.py'},
     {'chars': 1567,
      'ext': '.py',
      'lines': 49,
      'name': 'xoptim/patterns_exp/__init__.py'},
     {'chars': 1510,
      'ext': '.py',
      'lines': 47,
      'name': 'xoptim/patterns_exp/where_replace.py'},
     {'chars': 14628,
      'ext': '.py',
      'lines': 481,
      'name': 'xoptim/patterns_exp/binary_operators.py'},
     {'chars': 5007,
      'ext': '.py',
      'lines': 159,
      'name': 'xoptim/patterns/onnx_functions.py'},
     {'chars': 23482,
      'ext': '.py',
      'lines': 738,
      'name': 'xoptim/patterns/onnx_matmul.py'},
     {'chars': 5143,
      'ext': '.py',
      'lines': 183,
      'name': 'xoptim/patterns/onnx_any.py'},
     {'chars': 8928,
      'ext': '.py',
      'lines': 290,
      'name': 'xoptim/patterns/onnx_mul.py'},
     {'chars': 15883,
      'ext': '.py',
      'lines': 488,
      'name': 'xoptim/patterns/onnx_layer_normalization.py'},
     {'chars': 16531,
      'ext': '.py',
      'lines': 515,
      'name': 'xoptim/patterns/onnx_rotary.py'},
     {'chars': 11002,
      'ext': '.py',
      'lines': 341,
      'name': 'xoptim/patterns/onnx_reshape.py'},
     {'chars': 1609,
      'ext': '.py',
      'lines': 47,
      'name': 'xoptim/patterns/onnx_equal.py'},
     {'chars': 5496,
      'ext': '.py',
      'lines': 168,
      'name': 'xoptim/patterns/onnx_expand.py'},
     {'chars': 8901,
      'ext': '.py',
      'lines': 294,
      'name': 'xoptim/patterns/onnx_cast.py'},
     {'chars': 1426,
      'ext': '.py',
      'lines': 44,
      'name': 'xoptim/patterns/onnx_unsqueeze.py'},
     {'chars': 2571,
      'ext': '.py',
      'lines': 83,
      'name': 'xoptim/patterns/onnx_sub.py'},
     {'chars': 1077,
      'ext': '.py',
      'lines': 37,
      'name': 'xoptim/patterns/onnx_conv.py'},
     {'chars': 2432,
      'ext': '.py',
      'lines': 73,
      'name': 'xoptim/patterns/onnx_reduce.py'},
     {'chars': 3070,
      'ext': '.py',
      'lines': 102,
      'name': 'xoptim/patterns/onnx_split.py'},
     {'chars': 4061,
      'ext': '.py',
      'lines': 124,
      'name': 'xoptim/patterns/__init__.py'},
     {'chars': 6577,
      'ext': '.py',
      'lines': 244,
      'name': 'xoptim/patterns/onnx_transpose.py'},
     {'chars': 1136,
      'ext': '.py',
      'lines': 42,
      'name': 'xoptim/patterns/onnx_dropout.py'},
     {'chars': 4857,
      'ext': '.py',
      'lines': 151,
      'name': 'xoptim/patterns_ort/activation.py'},
     {'chars': 1456,
      'ext': '.py',
      'lines': 49,
      'name': 'xoptim/patterns_ort/gather_grad.py'},
     {'chars': 1929,
      'ext': '.py',
      'lines': 55,
      'name': 'xoptim/patterns_ort/activation_grad.py'},
     {'chars': 8732,
      'ext': '.py',
      'lines': 318,
      'name': 'xoptim/patterns_ort/fused_matmul.py'},
     {'chars': 4150,
      'ext': '.py',
      'lines': 117,
      'name': 'xoptim/patterns_ort/simplified_layer_normalization.py'},
     {'chars': 1127,
      'ext': '.py',
      'lines': 36,
      'name': 'xoptim/patterns_ort/__init__.py'},
     {'chars': 1283,
      'ext': '.py',
      'lines': 44,
      'name': 'xoptim/patterns_fix/add_reduction_scatter_nd.py'},
     {'chars': 429,
      'ext': '.py',
      'lines': 14,
      'name': 'xoptim/patterns_fix/__init__.py'},
     {'chars': 3533,
      'ext': '.py',
      'lines': 122,
      'name': 'torch_dynamo/dynger_backend.py'},
     {'chars': 9450,
      'ext': '.py',
      'lines': 326,
      'name': 'torch_dynamo/debug_backend.py'},
     {'chars': 19269,
      'ext': '.py',
      'lines': 636,
      'name': 'torch_dynamo/fast_backend.py'},
     {'chars': 2428,
      'ext': '.py',
      'lines': 72,
      'name': 'torch_dynamo/_dynamo_exporter.py'},
     {'chars': 4073,
      'ext': '.py',
      'lines': 128,
      'name': 'torch_dynamo/partition.py'},
     {'chars': 567,
      'ext': '.py',
      'lines': 18,
      'name': 'torch_dynamo/backend_helper.py'},
     {'chars': 6857,
      'ext': '.py',
      'lines': 225,
      'name': 'torch_dynamo/__init__.py'},
     {'chars': 5839,
      'ext': '.py',
      'lines': 165,
      'name': 'torch_bench/dort_profile.py'},
     {'chars': 976,
      'ext': '.py',
      'lines': 21,
      'name': 'torch_bench/bash_bench_huggingface_big.py'},
     {'chars': 17772,
      'ext': '.py',
      'lines': 607,
      'name': 'torch_bench/export_model_helper.py'},
     {'chars': 52587,
      'ext': '.py',
      'lines': 2033,
      'name': 'torch_bench/_bash_bench_benchmark_runner_agg_helper.py'},
     {'chars': 14125,
      'ext': '.py',
      'lines': 506,
      'name': 'torch_bench/_bash_bench_set_huggingface.py'},
     {'chars': 4087,
      'ext': '.py',
      'lines': 376,
      'name': 'torch_bench/_bash_bench_models_helper.py'},
     {'chars': 3068,
      'ext': '.py',
      'lines': 116,
      'name': 'torch_bench/_bash_bench_suites.py'},
     {'chars': 5466,
      'ext': '.py',
      'lines': 186,
      'name': 'torch_bench/bash_bench_agg.py'},
     {'chars': 31457,
      'ext': '.py',
      'lines': 1173,
      'name': 'torch_bench/_bash_bench_benchmark_runner_agg.py'},
     {'chars': 791,
      'ext': '.py',
      'lines': 19,
      'name': 'torch_bench/bash_bench_explicit.py'},
     {'chars': 5860,
      'ext': '.py',
      'lines': 164,
      'name': 'torch_bench/_bash_bench_set_dummies.py'},
     {'chars': 5975,
      'ext': '.py',
      'lines': 197,
      'name': 'torch_bench/dort_bench_profile.py'},
     {'chars': 16620,
      'ext': '.py',
      'lines': 526,
      'name': 'torch_bench/_dort_cmd_common.py'},
     {'chars': 892,
      'ext': '.py',
      'lines': 21,
      'name': 'torch_bench/bash_bench_issues.py'},
     {'chars': 2326,
      'ext': '.py',
      'lines': 89,
      'name': 'torch_bench/_bash_bench_set_issues.py'},
     {'chars': 972,
      'ext': '.py',
      'lines': 27,
      'name': 'torch_bench/bash_bench_torchbench.py'},
     {'chars': 2379,
      'ext': '.py',
      'lines': 91,
      'name': 'torch_bench/_bash_bench_set_explicit.py'},
     {'chars': 44847,
      'ext': '.py',
      'lines': 1469,
      'name': 'torch_bench/_bash_bench_benchmark_runner.py'},
     {'chars': 9819,
      'ext': '.py',
      'lines': 308,
      'name': 'torch_bench/dort_bench.py'},
     {'chars': 994,
      'ext': '.py',
      'lines': 27,
      'name': 'torch_bench/bash_bench_torchbench_ado.py'},
     {'chars': 44365,
      'ext': '.py',
      'lines': 1597,
      'name': 'torch_bench/_bash_bench_model_runner.py'},
     {'chars': 827,
      'ext': '.py',
      'lines': 19,
      'name': 'torch_bench/bash_bench_timm.py'},
     {'chars': 48, 'ext': '.py', 'lines': 1, 'name': 'torch_bench/__init__.py'},
     {'chars': 2675,
      'ext': '.py',
      'lines': 97,
      'name': 'torch_bench/_bash_bench_set_huggingface_big.py'},
     {'chars': 10000,
      'ext': '.py',
      'lines': 314,
      'name': 'torch_bench/_bash_bench_cmd.py'},
     {'chars': 13484,
      'ext': '.py',
      'lines': 532,
      'name': 'torch_bench/_bash_bench_set_timm.py'},
     {'chars': 7617,
      'ext': '.py',
      'lines': 264,
      'name': 'torch_bench/_dort_cmd_common_models.py'},
     {'chars': 1319,
      'ext': '.py',
      'lines': 43,
      'name': 'torch_bench/check_model.py'},
     {'chars': 1062,
      'ext': '.py',
      'lines': 23,
      'name': 'torch_bench/bash_bench_huggingface.py'},
     {'chars': 20453,
      'ext': '.py',
      'lines': 820,
      'name': 'torch_bench/_bash_bench_set_torchbench.py'},
     {'chars': 5300,
      'ext': '.py',
      'lines': 174,
      'name': 'torch_bench/export_model.py'},
     {'chars': 897,
      'ext': '.py',
      'lines': 34,
      'name': 'torch_bench/_bash_bench_set_torchbench_ado.py'},
     {'chars': 801,
      'ext': '.py',
      'lines': 19,
      'name': 'torch_bench/bash_bench_untrained.py'},
     {'chars': 685,
      'ext': '.py',
      'lines': 27,
      'name': 'torch_bench/big_models/try_minilm_test.py'},
     {'chars': 3273,
      'ext': '.py',
      'lines': 109,
      'name': 'torch_bench/big_models/try_minilm.py'},
     {'chars': 1825,
      'ext': '.py',
      'lines': 60,
      'name': 'torch_bench/big_models/try_falcon_mamba_test.py'},
     {'chars': 686,
      'ext': '.py',
      'lines': 27,
      'name': 'torch_bench/big_models/try_smollm_test.py'},
     {'chars': 2448,
      'ext': '.py',
      'lines': 93,
      'name': 'torch_bench/big_models/try_codellama.py'},
     {'chars': 2620,
      'ext': '.py',
      'lines': 93,
      'name': 'torch_bench/big_models/try_falcon_mamba.py'},
     {'chars': 914,
      'ext': '.py',
      'lines': 31,
      'name': 'torch_bench/big_models/try_flux_t5_test.py'},
     {'chars': 3076,
      'ext': '.py',
      'lines': 78,
      'name': 'torch_bench/big_models/main_phi_35_vision.py'},
     {'chars': 2922,
      'ext': '.py',
      'lines': 96,
      'name': 'torch_bench/big_models/try_stable_diffusion_3.py'},
     {'chars': 4514,
      'ext': '.py',
      'lines': 844,
      'name': 'torch_bench/big_models/try_phi_35_vision.py'},
     {'chars': 2755,
      'ext': '.py',
      'lines': 88,
      'name': 'torch_bench/big_models/try_phi_35_vision_test.py'},
     {'chars': 935,
      'ext': '.py',
      'lines': 32,
      'name': 'torch_bench/big_models/try_flux_transformer_test.py'},
     {'chars': 2349,
      'ext': '.py',
      'lines': 71,
      'name': 'torch_bench/big_models/try_phi_35_mini_instruct_test.py'},
     {'chars': 2339,
      'ext': '.py',
      'lines': 87,
      'name': 'torch_bench/big_models/try_smollm.py'},
     {'chars': 1455,
      'ext': '.py',
      'lines': 49,
      'name': 'torch_bench/big_models/try_phi_35_mini_instruct.py'},
     {'chars': 1649,
      'ext': '.py',
      'lines': 53,
      'name': 'torch_bench/big_models/__init__.py'},
     {'chars': 1864,
      'ext': '.py',
      'lines': 67,
      'name': 'torch_bench/big_models/try_flux_transformer.py'},
     {'chars': 692,
      'ext': '.py',
      'lines': 27,
      'name': 'torch_bench/big_models/try_codellama_test.py'},
     {'chars': 688,
      'ext': '.py',
      'lines': 26,
      'name': 'torch_bench/big_models/try_stable_diffusion_3_test.py'},
     {'chars': 1205,
      'ext': '.py',
      'lines': 47,
      'name': 'torch_bench/big_models/try_flux_t5.py'}]

experimental_experiment.ext_test_case.unit_test_going()

Enables a flag telling the script is running while testing it. Avois unit tests to be very long.

experimental_experiment.ext_test_case.with_path_append(path_to_add: str | List[str]) → Callable

Adds a path to sys.path to check.