MATH 697U: Introduction to Stochastic Processes, Spring 2010

Meeting : MWF 10:10--11:00    LGRT 123

Instructor : Luc Rey-Bellet

Office :  1423 J LGRT
Phone :  545-6020
E-Mail : luc <at> math.umass.edu
Office Hours :   Monday 2:00--3:15,   Wednesday 2:30--4:00,   or by appointment.

Classnotes Ch1--Ch2

Text: Introduction to Stochastic Processes, 2nd edition (2007) by Gregory F. Lawler, Chapman&Hall.

Further references:

• Introduction to Probability Models, 8-th Edition, by Sheldon M. Ross, Academic Press

• Lectures on Monte-Carlo Methods, by Neal N. Madras, American Mathematical Society

• Introduction to Stochastic Processes, by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.

• Stochastic Processes, by Sheldon M. Ross, Wiley.

• Adventures in Stochastic processes, by Sidney I. Resnick, Birkhauser.

• A first course in Stochastic Processes, by Samuel Karlin and Howard M. Taylor, Academic Press.

• An introduction to Markov processes. by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.

Syllabus: This course is an introduction to stochastic processes and Monte-Carlo methods. Prerequisite are a good knowledge of calculus and elementary probability as in Stat 515 or Stat 607. We present general concepts and techniques of the the theory of stochastic processes in particular Markov chains in discrete and continuous time. We emphasize examples from various disciplines, for example branching processes, queueing systems, population models, and so on. We will also discuss the numerical implementation of Markov chains and discuss the basics of Monte-Carlo algorithms. Among the topics treated in the class are

• Brief review of probability concepts. The limit theorems for sums of independent random variables.

• Simulation of random variables. A first look at Monte-Carlo algorithms.

• Markov chains on discrete state spaces (both finite and countable). Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, applications and examples.

• Continuous-Time Markov chains. Definition and basic properties. Poisson Process, Birth and Death Process, Queueing models.

• Reversible Markov processes and Monte-Carlo Markov algorithms.

• Martingales.

• Brownian motion and applications.

Grade :

• 1/3 Midterm (Take-home)

• 1/3 Final Exam

• 1/3 Homework, attendance and participation

Exams:

• Midterm: Midterm exam
  

• Final: Final exam
  

Homework :

• Homework #1 (due on February 15):   Homework #1
  

• Homework #2 (due on March 8 ):   Homework #2
  

• Homework #3 (due on April 19 )   Homework #3
  

• Homework #4 (due on May 3 ):   Homework #4