What is this?
This repository is a test of Unity’s ML-Agent plugin. In a nutshell, ML-Agents plugin lets you train AI using machine learning. The magic of intelligence happens in two steps.
In the first step you set-up an environment in which an AI body needs to figure out how to solve a situation. The situation is defined by a reward function. Roughly speaking, the reward function is positive when the AI body is doing well, it is negative when is doing badly. This environment is then used by an external algorithm to train. Hopefully the algorithm converges after some iterations. The result is a .bytes files that contains the pure essence of intelligence.
In the second step you implant the intelligence (.bytes file) in the AI body. After that, the AI body performs in a near-optimal way to solve the situation.
To understand how the ML-Agents plugin works I created the following problem:
What's the optimal way for a caterpillar to move?
I was expecting the AI to discover this:
So I created a model for a caterpillar with a single joint in the middle joining two points on the ground. This is how the caterpillar model moves when using random inputs:
After 250k iterations of the PPO algorithm that Unity provides out-of-the-box, the caterpillar learnt to move like this:
In the code and in other Unity assets I may refer to “caterpillar” as “slug”. That’s due to English not being my first language and me being lazy (sorry!).