Introduction to Stochastic Processes: Math 697U
Meeting : MWF 11:15--12:05 LGRT 121
Instructor : Luc Rey-Bellet
Office : 1423 J LGRT
Phone : 545-6020
E-Mail : luc@math.umass.edu
Office Hours :
Monday 1:30--3:00, Wednesday 2:30--4:00, or by appointment.
Classnotes :
Chapter 1.1--1.7 PDF file
Text:
Introduction to Stochastic Processes, 2nd edition (2007)
by Gregory F. Lawler, Chapman&Hall.
Further references:
-
Introduction to Probability Models,
8-th Edition, by Sheldon M. Ross, Academic Press
-
Lectures on Monte-Carlo Methods,
by Neal N. Madras, American Mathematical Society
-
Introduction to Stochastic Processes,
by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.
-
Stochastic Processes,
by Sheldon M. Ross, Wiley.
-
Adventures in Stochastic processes,
by Sidney I. Resnick, Birkhauser.
-
A first course in Stochastic Processes,
by Samuel Karlin and Howard M. Taylor, Academic Press.
-
An introduction to Markov processes.
by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.
Syllabus:
This course is an introduction to stochastic processes and Monte-Carlo methods.
Prerequisite are a good knowledge of calculus and elementary probability as
in Stat 515 or Stat 607. We present general concepts and techniques of the the theory of
stochastic processes in particular Markov chains in discrete and continuous time.
We emphasize examples from various disciplines, for example branching processes,
queueing systems, population models, and so on. We will also discuss the numerical
implementation of Markov chains and discuss the basics of Monte-Carlo algorithms.
Among the topics treated in the class are
-
Brief review of probability concepts. The limit theorems for sums of independent random variables. Simulation of random variables. A first look at Monte-Carlo algorithms.
-
Markov chains on discrete state spaces (both finite and
countable). Definition and basic properties, classification of states
(positive recurrence, recurrence and transience),
stationary distribution and limit theorems, analysis of
transient behavior, applications and examples.
-
Continuous-Time Markov chains. Definition and basic properties. Poisson
Process, Birth and Death Process, Queueing models.
-
Reversible Markov processes and Monte-Carlo Markov algorithms.
-
Brownian motion and applications.
Grade :
- 1/3 Midterm (Take-home)
- 1/3 Final Exam
- 1/3 Homework, attendance and participation
Exams:
Homework :