AnnaMats/ppo-Pyramids-Training

1# Background: PyTorch2 3As discussed in our4[machine learning background page](Background-Machine-Learning.md), many of the5algorithms we provide in the ML-Agents Toolkit leverage some form of deep6learning. More specifically, our implementations are built on top of the7open-source library [PyTorch](https://pytorch.org/). In this page we8provide a brief overview of PyTorch and TensorBoard9that we leverage within the ML-Agents Toolkit.10 11## PyTorch12 13[PyTorch](https://pytorch.org/) is an open source library for14performing computations using data flow graphs, the underlying representation of15deep learning models. It facilitates training and inference on CPUs and GPUs in16a desktop, server, or mobile device. Within the ML-Agents Toolkit, when you17train the behavior of an agent, the output is a model (.onnx) file that you can18then associate with an Agent. Unless you implement a new algorithm, the use of19PyTorch is mostly abstracted away and behind the scenes.20 21## TensorBoard22 23One component of training models with PyTorch is setting the values of24certain model attributes (called _hyperparameters_). Finding the right values of25these hyperparameters can require a few iterations. Consequently, we leverage a26visualization tool called27[TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard).28It allows the visualization of certain agent attributes (e.g. reward) throughout29training which can be helpful in both building intuitions for the different30hyperparameters and setting the optimal values for your Unity environment. We31provide more details on setting the hyperparameters in the32[Training ML-Agents](Training-ML-Agents.md) page. If you are unfamiliar with33TensorBoard we recommend our guide on34[using TensorBoard with ML-Agents](Using-Tensorboard.md) or this35[tutorial](https://github.com/dandelionmane/tf-dev-summit-tensorboard-tutorial).36