AnnaMats/ppo-Pyramids-Training

1# ML-Agents Versioning2 3## Context4As the ML-Agents project evolves into a more mature product, we want to communicate the process5we use to version our packages and the data that flows into, through, and out of them clearly.6Our project now has four packages (1 Unity, 3 Python) along with artifacts that are produced as7well as consumed.  This document covers the versioning for these packages and artifacts.8 9## GitHub Releases10Up until now, all packages were in lockstep in-terms of versioning. As a result, the GitHub releases11were tagged with the version of all those packages (e.g. v0.15.0, v0.15.1) and labeled accordingly.12With the decoupling of package versions, we now need to revisit our GitHub release tagging.13The proposal is that we move towards an integer release numbering for our repo and each such14release will call out specific version upgrades of each package. For instance, with15[the April 30th release](https://github.com/Unity-Technologies/ml-agents/releases/tag/release_1),16we will have:17- GitHub Release 1 (branch name: *release_1_branch*)18  - com.unity.ml-agents release 1.0.019  - ml-agents release 0.16.020  - ml-agents-envs release 0.16.021  - gym-unity release 0.16.022 23Our release cadence will not be affected by these versioning changes.  We will keep having24monthly releases to fix bugs and release new features.25 26## Packages27All of the software packages, and their generated artifacts will be versioned.  Any automation28tools will not be versioned.29 30### Unity package31Package name: com.unity.ml-agents32- Versioned following [Semantic Versioning Guidelines](https://www.semver.org)33- This package consumes an artifact of the training process: the `.nn` file.  These files34    are integer versioned and currently at version 2. The com.unity.ml-agents package35    will need to support the version of `.nn` files which existed at its 1.0.0 release.36    For example, consider that com.unity.ml-agents is at version 1.0.0 and the NN files37    are at version 2.  If the NN files change to version 3, the next release of38    com.unity.ml-agents at version 1.1.0 guarantees it will be able to read both of these39    formats.  If the NN files were to change to version 4 and com.unity.ml-agents to40    version 2.0.0, support for NN versions 2 and 3 could be dropped for com.unity.ml-agents41    version 2.0.0.42- This package produces one artifact, the `.demo` files.  These files will have integer43    versioning. This means their version will increment by 1 at each change.  The44    com.unity.ml-agents package must be backward compatible with version changes45    that occur between minor versions.46- To summarize, the artifacts produced and consumed by com.unity.ml-agents are guaranteed47    to be supported for 1.x.x versions of com.unity.ml-agents.  We intend to provide stability48    for our users by moving to a 1.0.0 release of com.unity.ml-agents.49 50 51### Python Packages52Package names: ml-agents / ml-agents-envs / gym-unity53- The python packages remain in "Beta."  This means that breaking changes to the public54    API of the python packages can change without having to have a major version bump.55    Historically, the python and C# packages were in version lockstep.  This is no longer56    the case.  The python packages will remain in lockstep with each other for now, while the57    C# package will follow its own versioning as is appropriate.  However, the python package58    versions may diverge in the future.59- While the python packages will remain in Beta for now, we acknowledge that the most60    heavily used portion of our python interface is the `mlagents-learn` CLI and strive61    to make this part of our API backward compatible. We are actively working on this and62    expect to have a stable CLI in the next few weeks.63 64## Communicator65 66Packages which communicate: com.unity.ml-agents / ml-agents-envs67 68Another entity of the ML-Agents Toolkit that requires versioning is the communication layer69between C# and Python, which will follow also semantic versioning.  This guarantees a level of70backward compatibility between different versions of C# and Python packages which communicate.71Any Communicator version 1.x.x of the Unity package should be compatible with any 1.x.x72Communicator Version in Python.73 74An RLCapabilities struct keeps track of which features exist. This struct is passed from C# to75Python, and another from Python to C#.  With this feature level granularity, we can notify users76more specifically about feature limitations based on what's available in both C# and Python.77These notifications will be logged to the python terminal, or to the Unity Editor Console.78 79 80## Side Channels81 82The communicator is what manages data transfer between Unity and Python for the core83training loop. Side Channels are another means of data transfer between Unity and Python.84Side Channels are not versioned, but have been designed to support backward compatibility85for what they are. As of today, we provide 4 side channels:86- FloatProperties: shared float data between Unity - Python (bidirectional)87- RawBytes: raw data that can be sent Unity - Python (bidirectional)88- EngineConfig: a set of numeric fields in a pre-defined order sent from Python to Unity89- Stats: (name, value, agg) messages sent from Unity to Python90 91Aside from the specific implementations of side channels we provide (and use ourselves),92the Side Channel interface is made available for users to create their own custom side93channels. As such, we guarantee that the built in SideChannel interface between Unity and94Python is backward compatible in packages that share the same major version.95 96