Yes, it is possible to use OpenAI gym environments for multi-agent games. Although in the OpenAI gym community there is no standardized interface for multi-agent environments, it is easy enough to build an OpenAI gym that supports this. For instance, in OpenAI’s recent work on multi-agent particle environments they make a multi-agent environment that inherits … Read more
Are SVMs better than ANN with many classes? You are probably referring to the fact that SVMs are in essence, either either one-class or two-class classifiers. Indeed they are and there’s no way to modify a SVM algorithm to classify more than two classes. The fundamental feature of a SVM is the separating maximum-margin hyperplane … Read more
Reinforcement Learning In reinforcement learning, an agent tries to come up with the best action given a state. For example, in the video game Pac-Man, the state space would be the 2D game world you are in, the surrounding items (pac-dots, enemies, walls, etc), and actions would be moving through that 2D space (going up/down/left/right). … Read more
Box means that you are dealing with real valued quantities. The first array np.array([-1,0,0] are the lowest accepted values, and the second np.array([+1,+1,+1]) are the highest accepted values. In this case (using the comment) we see that we have 3 available actions: Steering: Real valued in [-1, 1] Gas: Real valued in [0, 1] Brake: … Read more
You are 100% right that if we knew the transition probabilities and reward for every transition in Q-learning, it would be pretty unclear why we would use it instead of model-based learning or how it would even be fundamentally different. After all, transition probabilities and rewards are the two components of the model used in … Read more
The definition is correct, though not instantly obvious if you see it for the first time. Let me put it this way: a policy is an agent’s strategy. For example, imagine a world where a robot moves across the room and the task is to get to the target point (x, y), where it gets … Read more
The common way of dealing with this problem is with actor-critic methods. These naturally extend to continuous action spaces. Basic Q-learning could diverge when working with approximations, however, if you still want to use it, you can try combining it with a self-organizing map, as done in “Applications of the self-organising map to reinforcement learning”. … Read more
To better understand PPO, it is helpful to look at the main contributions of the paper, which are: (1) the Clipped Surrogate Objective and (2) the use of “multiple epochs of stochastic gradient ascent to perform each policy update”. From the original PPO paper: We have introduced [PPO], a family of policy optimization methods that … Read more
There are some research papers on the topic: Efficient Reinforcement Learning Through Evolving Neural Network Topologies (2002) Reinforcement Learning Using Neural Networks, with Applications to Motor Control Reinforcement Learning Neural Network To The Problem Of Autonomous Mobile Robot Obstacle Avoidance And some code: Code examples for neural network reinforcement learning. Those are just some of … Read more
Let’s look at them side by side. The key parts for comparison are highlighted. Figures are from Sutton and Barto’s book: Reinforcement Learning: An Introduction. Key points: Policy iteration includes: policy evaluation + policy improvement, and the two are repeated iteratively until policy converges. Value iteration includes: finding optimal value function + one policy extraction. … Read more