So in the next few weeks I will be releasing a minor update to Shodan, my Q-learning library. This change will make it possible for Shodan to interact with real time engines like Box2D.

To demonstrate, I have created a simple demo. In the Box2D world there is a robot created out of three rectangles: the body, the arm, and the hand. These are connected to each other via revolute joints. Shodan is tasked with controlling the motor of these joints with the goal of moving the robot in either a positive or negative x position depending on the reward set. In other words, to train the robot to learn to crawl. Read more

At last!! I am releasing Shodan, my implementation of the Q-learning algorithm. A quick recap. Q-learning is a type of reinforcement learning, you could consider it a type of Artificial Intelligence but it actually falls under a category called Machine Learning. What makes Q-learning so cool is that it can learn and adapt to achieve its goals. That means, if we created a robot with an arm, after training, it could potentially learn the most optimal way to crawl. Exciting stuff. For more info on Q-learning refer to my earlier post.

You can grab all the files at my repo on Github. Enjoy

PS – For anyone wondering why I called my library Shodan, google the game System Shock 2

Q-learning is a reinforcement learning algorithm that falls under the umbrella of machine learning, an area closely related to artificial intelligence.

Unlike a neural network which is a supervised learning algorithm, Q-learning learns independently by acting on observations of its state. For example: we might send a mars rover-esque robot to another planet which has an unknown surface. With reinforcement learning the robot could potentially learn about the surface through trial and error and use this new found knowledge in an effort to best reach its goals (for example: the most efficient way to move across the surface).

In my opinion this type of learning best resembles how we as humans learn.

How does Q-learning work? Read more