markov – Tarik Billa

Markov decision process value iteration, how does it work?

August 9, 2023 by Tarik

Yes, the mathematical notation can make it seem much more complicated than it is. Really, it is a very simple idea. I have a implemented a value iteration demo applet that you can play with to get a better idea. Basically, lets say you have a 2D grid with a robot in it. The robot … Read more

What is the difference between markov chains and hidden markov model?

March 26, 2023 by Tarik

To explain by example, I’ll use an example from natural language processing. Imagine you want to know the probability of this sentence: I enjoy coffee In a Markov model, you could estimate its probability by calculating: P(WORD = I) x P(WORD = enjoy | PREVIOUS_WORD = I) x P(word = coffee| PREVIOUS_WORD = enjoy) Now, … Read more