Markov chains can be represented by finite state machines. The idea is that a Markov chain describes a process in which the transition to a state at time t+1 depends only on the state at time t. The main thing to keep in mind is that the transitions in a Markov chain are probabilistic rather than deterministic, which means that you can’t always say with perfect certainty what will happen at time t+1.
The Wikipedia articles on Finite-state machines has a subsection on Finite Markov-chain processes, I’d recommend reading that for more information. Also, the Wikipedia article on Markov chains has a brief sentence describing the use of finite state machines in representing a Markov chain. That states:
A finite state machine can be used as
a representation of a Markov chain.
Assuming a sequence of independent and
identically distributed input signals
(for example, symbols from a binary
alphabet chosen by coin tosses), if
the machine is in state y at time n,
then the probability that it moves to
state x at time n + 1 depends only on
the current state.