AnnaMats/ppo-Pyramids-Training

1# Custom Side Channels2 3You can create your own side channel in C# and Python and use it to communicate4custom data structures between the two. This can be useful for situations in5which the data to be sent is too complex or structured for the built-in6`EnvironmentParameters`, or is not related to any specific agent, and therefore7inappropriate as an agent observation.8 9## Overview10 11In order to use a side channel, it must be implemented as both Unity and Python12classes.13 14### Unity side15 16The side channel will have to implement the `SideChannel` abstract class and the17following method.18 19- `OnMessageReceived(IncomingMessage msg)` : You must implement this method and20  read the data from IncomingMessage. The data must be read in the order that it21  was written.22 23The side channel must also assign a `ChannelId` property in the constructor. The24`ChannelId` is a Guid (or UUID in Python) used to uniquely identify a side25channel. This Guid must be the same on C# and Python. There can only be one side26channel of a certain id during communication.27 28To send data from C# to Python, create an `OutgoingMessage` instance, add data29to it, call the `base.QueueMessageToSend(msg)` method inside the side channel,30and call the `OutgoingMessage.Dispose()` method.31 32To register a side channel on the Unity side, call33`SideChannelManager.RegisterSideChannel` with the side channel as only argument.34 35### Python side36 37The side channel will have to implement the `SideChannel` abstract class. You38must implement :39 40- `on_message_received(self, msg: "IncomingMessage") -> None` : You must41  implement this method and read the data from IncomingMessage. The data must be42  read in the order that it was written.43 44The side channel must also assign a `channel_id` property in the constructor.45The `channel_id` is a UUID (referred in C# as Guid) used to uniquely identify a46side channel. This number must be the same on C# and Python. There can only be47one side channel of a certain id during communication.48 49To assign the `channel_id` call the abstract class constructor with the50appropriate `channel_id` as follows:51 52```python53super().__init__(my_channel_id)54```55 56To send a byte array from Python to C#, create an `OutgoingMessage` instance,57add data to it, and call the `super().queue_message_to_send(msg)` method inside58the side channel.59 60To register a side channel on the Python side, pass the side channel as argument61when creating the `UnityEnvironment` object. One of the arguments of the62constructor (`side_channels`) is a list of side channels.63 64## Example implementation65 66Below is a simple implementation of a side channel that will exchange ASCII67encoded strings between a Unity environment and Python.68 69### Example Unity C# code70 71The first step is to create the `StringLogSideChannel` class within the Unity72project. Here is an implementation of a `StringLogSideChannel` that will listen73for messages from python and print them to the Unity debug log, as well as send74error messages from Unity to python.75 76```csharp77using UnityEngine;78using Unity.MLAgents;79using Unity.MLAgents.SideChannels;80using System.Text;81using System;82 83public class StringLogSideChannel : SideChannel84{85    public StringLogSideChannel()86    {87        ChannelId = new Guid("621f0a70-4f87-11ea-a6bf-784f4387d1f7");88    }89 90    protected override void OnMessageReceived(IncomingMessage msg)91    {92        var receivedString = msg.ReadString();93        Debug.Log("From Python : " + receivedString);94    }95 96    public void SendDebugStatementToPython(string logString, string stackTrace, LogType type)97    {98        if (type == LogType.Error)99        {100            var stringToSend = type.ToString() + ": " + logString + "\n" + stackTrace;101            using (var msgOut = new OutgoingMessage())102            {103                msgOut.WriteString(stringToSend);104                QueueMessageToSend(msgOut);105            }106        }107    }108}109```110 111Once we have defined our custom side channel class, we need to ensure that it is112instantiated and registered. This can typically be done wherever the logic of113the side channel makes sense to be associated, for example on a MonoBehaviour114object that might need to access data from the side channel. Here we show a115simple MonoBehaviour object which instantiates and registers the new side116channel. If you have not done it already, make sure that the MonoBehaviour which117registers the side channel is attached to a GameObject which will be live in118your Unity scene.119 120```csharp121using UnityEngine;122using Unity.MLAgents;123 124 125public class RegisterStringLogSideChannel : MonoBehaviour126{127 128    StringLogSideChannel stringChannel;129    public void Awake()130    {131        // We create the Side Channel132        stringChannel = new StringLogSideChannel();133 134        // When a Debug.Log message is created, we send it to the stringChannel135        Application.logMessageReceived += stringChannel.SendDebugStatementToPython;136 137        // The channel must be registered with the SideChannelManager class138        SideChannelManager.RegisterSideChannel(stringChannel);139    }140 141    public void OnDestroy()142    {143        // De-register the Debug.Log callback144        Application.logMessageReceived -= stringChannel.SendDebugStatementToPython;145        if (Academy.IsInitialized){146            SideChannelManager.UnregisterSideChannel(stringChannel);147        }148    }149 150    public void Update()151    {152        // Optional : If the space bar is pressed, raise an error !153        if (Input.GetKeyDown(KeyCode.Space))154        {155            Debug.LogError("This is a fake error. Space bar was pressed in Unity.");156        }157    }158}159```160 161### Example Python code162 163Now that we have created the necessary Unity C# classes, we can create their164Python counterparts.165 166```python167from mlagents_envs.environment import UnityEnvironment168from mlagents_envs.side_channel.side_channel import (169    SideChannel,170    IncomingMessage,171    OutgoingMessage,172)173import numpy as np174import uuid175 176 177# Create the StringLogChannel class178class StringLogChannel(SideChannel):179 180    def __init__(self) -> None:181        super().__init__(uuid.UUID("621f0a70-4f87-11ea-a6bf-784f4387d1f7"))182 183    def on_message_received(self, msg: IncomingMessage) -> None:184        """185        Note: We must implement this method of the SideChannel interface to186        receive messages from Unity187        """188        # We simply read a string from the message and print it.189        print(msg.read_string())190 191    def send_string(self, data: str) -> None:192        # Add the string to an OutgoingMessage193        msg = OutgoingMessage()194        msg.write_string(data)195        # We call this method to queue the data we want to send196        super().queue_message_to_send(msg)197```198 199We can then instantiate the new side channel, launch a `UnityEnvironment` with200that side channel active, and send a series of messages to the Unity environment201from Python using it.202 203```python204# Create the channel205string_log = StringLogChannel()206 207# We start the communication with the Unity Editor and pass the string_log side channel as input208env = UnityEnvironment(side_channels=[string_log])209env.reset()210string_log.send_string("The environment was reset")211 212group_name = list(env.behavior_specs.keys())[0]  # Get the first group_name213group_spec = env.behavior_specs[group_name]214for i in range(1000):215    decision_steps, terminal_steps = env.get_steps(group_name)216    # We send data to Unity : A string with the number of Agent at each217    string_log.send_string(218        f"Step {i} occurred with {len(decision_steps)} deciding agents and "219        f"{len(terminal_steps)} terminal agents"220    )221    env.step()  # Move the simulation forward222 223env.close()224```225 226Now, if you run this script and press `Play` the Unity Editor when prompted, the227console in the Unity Editor will display a message at every Python step.228Additionally, if you press the Space Bar in the Unity Engine, a message will229appear in the terminal.230