One
may consider the Markov chain a walk or path through phase space. The label k, which
sequences the states in the Markov chain, can be thought of as time. Then starting from
initial state
the Monte Carlo procedure
generates time ordered states![](Image23.gif)
Due to the construction the states will ultimately be distributed in some way. What we
need to specify is that the distribution of the states is guaranteed to be the
distribution in thermal equilibrium. Before developing the necessary conditions to ensure
that the transitions from one state to the next yield the correct distribution, let us
dwell on the idea that we generate a trajectory in configuration space.
The definition for the time-dependent average of an observable is
![](Image24.gif)
where P(x,t) is the time-dependent probability density for the states. It can be
shown that
![](Image25.gif)
i.e., one may average over the quantity one is interested in along a trajectory
generated by the Monte Carlo method. This is similar to the trajectories generated in
molecular dynamics simulations described in the previous chapter.
The Markov chain is constructed such that the states are distributed as in thermal
equilibrium, i.e., here with the canonical distribution
![](Image26.gif)
If one has constructed transition probabilities from one state to another which give
the distribution
![](Image27.gif)
one obtains for the observable A
![](Image28.gif)
i.e., again a simple arithmetic average. However, we have not performed a simple
sampling but an importance sampling. This is because the states that we use to
sample the observable are generated with the correct distribution!