Reinforcement Learning In reinforcement learning, an agent tries to come up with the best action given a state. For example, in the video game Pac-Man, the state space would be the 2D game world you are in, the surrounding items (pac-dots, enemies, walls, etc), and actions would be moving through that 2D space (going up/down/left/right). … Read more
You are 100% right that if we knew the transition probabilities and reward for every transition in Q-learning, it would be pretty unclear why we would use it instead of model-based learning or how it would even be fundamentally different. After all, transition probabilities and rewards are the two components of the model used in … Read more
The common way of dealing with this problem is with actor-critic methods. These naturally extend to continuous action spaces. Basic Q-learning could diverge when working with approximations, however, if you still want to use it, you can try combining it with a self-organizing map, as done in “Applications of the self-organising map to reinforcement learning”. … Read more
When I was learning this part, I found it very confusing too, so I put together the two pseudo-codes from R.Sutton and A.G.Barto hoping to make the difference clearer. Blue boxes highlight the part where the two algorithms actually differ. Numbers highlight the more detailed difference to be explained later. TL;NR: | | SARSA | … Read more