Excel spreadsheet is probably the most popular medium for implementing Markov models. I learned the approach from a step-by-step guide by Damon Douglas (published through Knol, a service from Google and now discontinued). I tried to translate the guide into a script for R. Any comments or suggestions for improvement are welcome.
horizon<-10 # model horizon (years, days, whatever)
n=1000 # population size for a population matrix
sim_n<-1000 # number of simulations
n_strat<-2 # number of strategies
n_states<-3 # number of states