experimental_experiment.torch_bench._dort_cmd_common¶
- experimental_experiment.torch_bench._dort_cmd_common.create_compiled_model(model: Any, backend: str, target_opset: int, use_dynamic: bool = False, verbose: int = 0, enable_pattern: str | List[str] = 'default', disable_pattern: str | List[str] | None = None, return_storage: bool = False, rename_inputs: bool = True, dump_prefix: str | None = None, dump_patterns: str | None = None, optimize: bool = True, ort_optimize: bool = True, use_fused_aten_ops: bool = False, processor: str = 'CPU', order_algorithm: str = 'NONE') Any[source]¶
Creates the compiled model.
- Parameters:
model – module
backend – kind of backend
use_dynamic – use dynamic shape
verbose – verbosity
enable_pattern – to enable optimization pattern
disable_pattern – to disable optimization pattern
return_storage – return a container for the models, only works with backend custom and debug
rename_inputs – rename inputs into
input_{i}dump_prefix – dumps the models (backend, custom and debug)
dump_patterns – dumps the optimization applied patterns if applicable
optimize – enable optimizations
ort_optimize – enables onnxruntime optimization
use_fused_aten_ops – use fused opetor when converting the model, it only works the backend custom
processor – optimization should be made for this processor or this list of processors (comma separated value)
order_algorithm – algorithm optimizing the order the onnx node, none by default
- Returns:
compiled model
- experimental_experiment.torch_bench._dort_cmd_common.create_configuration_for_benchmark(model: str = 'llama', config: str = 'small', repeat: int = 5, warmup: int = 3, num_hidden_layers: int = 1, implementation: str = 'eager', with_mask: bool = True, shape_scenario: str | None = None, dynamic_shapes: bool = False, dtype: str = 'float32') Dict[str, str | int | List[Tuple[int, int]]][source]¶
Creates a model based on the given configuration.
- Parameters:
model – model name
config – size of the model (small, medium, large)
warmup – number of warmup steps
repeat – number of repetition
num_hidden_layers – number of hidden layers
implementation – implementation
with_mask – use a mask
shape_scenario – None or empty for all shapes equal to (2, 1024), ‘batch’ for different batch sizes, ‘length’ for different length sizes
dynamic_shapes – use dynamic shapes
- Returns:
dictionary
- experimental_experiment.torch_bench._dort_cmd_common.create_model(model: str, config_dict: Dict[str, int | str], dtype: str | None = 'float32') Tuple[Any, List[Tuple[Any, ...]]][source]¶
Returns a model and a list of inputs.
- Parameters:
model – model name
config_dict – configuration
dtype – dtype to use
- Returns:
model, list of inputs