Introduction to Stochastic Processes: Math 697U
Meeting : MWF 11:1512:05 LGRT 121
Instructor : Luc ReyBellet
Office : 1423 J LGRT
Phone : 5456020
EMail : luc@math.umass.edu
Office Hours :
Monday 1:303:00, Wednesday 2:304:00, or by appointment.
Classnotes :
Chapter 1.11.7 PDF file
Text:
Introduction to Stochastic Processes, 2nd edition (2007)
by Gregory F. Lawler, Chapman&Hall.
Further references:

Introduction to Probability Models,
8th Edition, by Sheldon M. Ross, Academic Press

Lectures on MonteCarlo Methods,
by Neal N. Madras, American Mathematical Society

Introduction to Stochastic Processes,
by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.

Stochastic Processes,
by Sheldon M. Ross, Wiley.

Adventures in Stochastic processes,
by Sidney I. Resnick, Birkhauser.

A first course in Stochastic Processes,
by Samuel Karlin and Howard M. Taylor, Academic Press.

An introduction to Markov processes.
by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.
Syllabus:
This course is an introduction to stochastic processes and MonteCarlo methods.
Prerequisite are a good knowledge of calculus and elementary probability as
in Stat 515 or Stat 607. We present general concepts and techniques of the the theory of
stochastic processes in particular Markov chains in discrete and continuous time.
We emphasize examples from various disciplines, for example branching processes,
queueing systems, population models, and so on. We will also discuss the numerical
implementation of Markov chains and discuss the basics of MonteCarlo algorithms.
Among the topics treated in the class are

Brief review of probability concepts. The limit theorems for sums of independent random variables. Simulation of random variables. A first look at MonteCarlo algorithms.

Markov chains on discrete state spaces (both finite and
countable). Definition and basic properties, classification of states
(positive recurrence, recurrence and transience),
stationary distribution and limit theorems, analysis of
transient behavior, applications and examples.

ContinuousTime Markov chains. Definition and basic properties. Poisson
Process, Birth and Death Process, Queueing models.

Reversible Markov processes and MonteCarlo Markov algorithms.

Brownian motion and applications.
Grade :
 1/3 Midterm (Takehome)
 1/3 Final Exam
 1/3 Homework, attendance and participation
Exams:
Homework :