tools

decompress_zip

teachpyx.tools.decompress_zip(filename, dest: str, verbose: bool = False) → List[str]

Unzips a zip file.

Paramètres:

• filename – file to process

• dest – destination

• verbose – verbosity

Renvoie:

return the list of decompressed files

download_and_unzip

teachpyx.tools.download_and_unzip(url: str, dest: str = '.', timeout: int = 10, verbose: bool = False) → List[str]

Downloads a file and unzip it.

Paramètres:

• url – url

• dest – destination folder

• timeout – timeout

• verbose – display progress

Renvoie:

list of unzipped files

draw_diagram

teachpyx.tools.draw_diagram(graph, module='blockdiag', fmt='pillow', **options)

Draws a graph based on module blockdiag.

Paramètres:

• graph – definition, see blockdiag

• module – 'blockdiag' (only available value)

• fmt – can be a filename or a module name ('pillow')

• options – additional options for blockdiag

Renvoie:

graph

blockdiag {
    A -> B -> C -> D;
    A -> E -> F -> G;
}

total_size

teachpyx.tools.total_size(o, handlers=None)

Returns the approximate memory footprint an object and all of its contents.

Paramètres:

• o – object to measure

• handlers – for recursivity purpose

Renvoie:

total size

Automatically finds the contents of the following builtin containers and their subclasses: tuple, list, deque, dict, set and frozenset. To search other containers, add handlers to iterate over their contents:

handlers = {SomeContainerClass: iter,
            OtherContainerClass: OtherContainerClass.get_elements}

Source : activestate referenced by function getsizeof.

profiling

class teachpyx.tools.profiling.ProfileNode(filename: str, line: int, func_name: str, nc1: int, nc2: int, tin: float, tall: float)

Graph structure to represent a profiling.

Paramètres:

• filename – filename

• line – line number

• func_name – function name

• nc1 – number of calls 1

• nc2 – number of calls 2

• tin – time spent in the function

• tout – time spent in the function and in the sub functions

add_called_by(pnode: ProfileNode)

This function is called by these lines.

add_calls_to(pnode: ProfileNode, time_elements)

This function calls this node.

as_dict(filter_node=None, sort_key=SortKey.LINE)

Renders the results of a profiling interpreted with function @fn profile2graph. It can then be loaded with a dataframe.

Paramètres:

• filter_node – display only the nodes for which this function returns True, if None, the default function removes built-in function with small impact

• sort_key – sort sub nodes by…

Renvoie:

rows

static filter_node_(node, info=None) → bool

Filters out node to be displayed by default.

Paramètres:

• node – node

• info – if the node is called by a function, this dictionary can be used to overwrite the attributes held by the node

Renvoie:

boolean (True to keep, False to forget)

get_root()

Returns the root of the graph.

property key

Returns file:line.

to_json(filter_node=None, sort_key=SortKey.LINE, as_str=True, **kwargs) → str

Renders the results of a profiling interpreted with function @fn profile2graph as JSON.

Paramètres:

• filter_node – display only the nodes for which this function returns True, if None, the default function removes built-in function with small impact

• sort_key – sort sub nodes by…

• as_str – converts the json into a string

• kwargs – see json.dumps()

Renvoie:

rows

to_text(filter_node=None, sort_key=SortKey.LINE, fct_width=60) → str

Prints the profiling to text.

Paramètres:

• filter_node – display only the nodes for which this function returns True, if None, the default function removes built-in function with small impact

• sort_key – sort sub nodes by…

Renvoie:

rows

teachpyx.tools.profiling.profile(fct: Callable, sort: str = 'cumulative', rootrem: str | None = None, as_df: bool = False, return_results: bool = False, **kwargs) → str

Profiles the execution of a function.

Paramètres:

• fct – function to profile

• sort – see pstats.Stats.sort_stats()

• rootrem – root to remove in filenames

• as_df – return the results as a dataframe and not text

• return_results – if True, return results as well (in the first position)

• kwargs – additional parameters used to create the profiler, see cProfile.Profile

Renvoie:

raw results, statistics text dump (or dataframe is as_df is True)

(Source code, png, hires.png, pdf)

teachpyx.tools.profiling.profile2df(ps: Stats, as_df: bool = True, clean_text: bool | None = None, verbose: bool = False, logf=None)

Converts profiling statistics into a Dataframe.

Paramètres:

• ps – an instance of pstats

• as_df – returns the results as a dataframe (True) or a list of dictionaries (False)

• clean_text – function to clean function names

• verbose – verbosity

• logf – logging function

Renvoie:

a DataFrame

import pstats
from onnx_array_api.profiling import profile2df

ps = pstats.Stats('bench_ortmodule_nn_gpu6.prof')
df = profile2df(pd)
print(df)

teachpyx.tools.profiling.profile2graph(ps: Stats, clean_text: Callable | None = None, verbose: bool = False, logf: Callable | None = None) → ProfileNode

Converts profiling statistics into a graphs.

Paramètres:

• ps –

  an instance of pstats

• clean_text – function to clean function names

• verbose – verbosity

• logf – logging function

Renvoie:

an instance of class @see cl ProfileNode

pyinstrument has a nice display to show time spent and call stack at the same time. This function tries to replicate that display based on the results produced by module cProfile. Here is an example.

<<<

import time
from teachpyx.tools.profiling import profile, profile2graph


def fct0(t):
    time.sleep(t)


def fct1(t):
    time.sleep(t)


def fct2():
    fct1(0.1)
    fct1(0.01)


def fct3():
    fct0(0.2)
    fct1(0.5)


def fct4():
    fct2()
    fct3()


ps = profile(fct4)[0]
root, nodes = profile2graph(ps, clean_text=lambda x: x.split("/")[-1])
text = root.to_text()
print(text)

>>>

    fct1                                                         --  3  3 -- 0.00003 0.61100 -- :11:fct1 (fct1)
        <built-in method time.sleep>                             --  3  3 -- 0.61097 0.61097 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>) +++
    fct4                                                         --  1  1 -- 0.00001 0.81140 -- :25:fct4 (fct4)
        fct2                                                     --  1  1 -- 0.00001 0.11034 -- :15:fct2 (fct2)
            fct1                                                 --  2  2 -- 0.00002 0.11033 -- :11:fct1 (fct1) +++
        fct3                                                     --  1  1 -- 0.00001 0.70105 -- :20:fct3 (fct3)
            fct0                                                 --  1  1 -- 0.00001 0.20037 -- :7:fct0 (fct0)
                <built-in method time.sleep>                     --  1  1 -- 0.20036 0.20036 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>) +++
            fct1                                                 --  1  1 -- 0.00001 0.50067 -- :11:fct1 (fct1) +++
    <built-in method time.sleep>                                 --  4  4 -- 0.81132 0.81132 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>)