AnnaMats/ppo-Pyramids-Training

1# Profiling in Python2 3As part of the ML-Agents Tookit, we provide a lightweight profiling system, in4order to identity hotspots in the training process and help spot regressions5from changes.6 7Timers are hierarchical, meaning that the time tracked in a block of code can be8further split into other blocks if desired. This also means that a function that9is called from multiple places in the code will appear in multiple places in the10timing output.11 12All timers operate using a "global" instance by default, but this can be13overridden if necessary (mainly for testing).14 15## Adding Profiling16 17There are two ways to indicate code should be included in profiling. The18simplest way is to add the `@timed` decorator to a function or method of19interested.20 21```python22class TrainerController:23    # ....24    @timed25    def advance(self, env: EnvManager) -> int:26        # do stuff27```28 29You can also used the `hierarchical_timer` context manager.30 31```python32with hierarchical_timer("communicator.exchange"):33    outputs = self.communicator.exchange(step_input)34```35 36The context manager may be easier than the `@timed` decorator for profiling37different parts of a large function, or profiling calls to abstract methods that38might not use decorator.39 40## Output41 42By default, at the end of training, timers are collected and written in json43format to `{summaries_dir}/{run_id}_timers.json`. The output consists of node44objects with the following keys:45 46- total (float): The total time in seconds spent in the block, including child47  calls.48- count (int): The number of times the block was called.49- self (float): The total time in seconds spent in the block, excluding child50  calls.51- children (dictionary): A dictionary of child nodes, keyed by the node name.52- is_parallel (bool): Indicates that the block of code was executed in multiple53  threads or processes (see below). This is optional and defaults to false.54 55### Parallel execution56 57#### Subprocesses58 59For code that executes in multiple processes (for example,60SubprocessEnvManager), we periodically send the timer information back to the61"main" process, aggregate the timers there, and flush them in the subprocess.62Note that (depending on the number of processes) this can result in timers where63the total time may exceed the parent's total time. This is analogous to the64difference between "real" and "user" values reported from the unix `time`65command. In the timer output, blocks that were run in parallel are indicated by66the `is_parallel` flag.67 68#### Threads69 70Timers currently use `time.perf_counter()` to track time spent, which may not71give accurate results for multiple threads. If this is problematic, set72`threaded: false` in your trainer configuration.73