**MATH 697U: Stochastic Processes and Applications, Spring 2018**

**Meeting :**
TuTh 10:00--11:15 LGRT 145

**Instructor :** Luc
Rey-Bellet

**Office :** 1423 K LGRT

**Phone :** 545-6020

**E-Mail : **luc <at> math.umass.edu

**Office Hours :** Tu 2:00-3:30, Th 1:00--2:15, or by appointment.

** Classnotes: **

- Classnotes are available here and will be regularly updated: Classnotes

**Text**: There is no official textbook for the class.

The book by Lawler is an excellent and very efficient introductory textbook which will cover a large part of the class. It is highly recommended for a first read to have a broad overview.

The book by Sheldon Ross is more elementary elementary one. It has a lot of very good examples.
I used part of chapter 11 on simulation to prepare for my class.

The books by Resnick and Durrett are more advanced and are recommended for the more mathematically oriented student.

The books by Madras and Rubinstein-Kroese are about Monte-Carlo methods and Simulation.

The book by Levin, Peres and Wilmer is about the modern theory of finite state Markov chains. I am using part of the first four chapters of the book for our class.

**References:**

*Introduction to Stochastic Processes,*2nd edition (2007) by Gregory F. Lawler, Chapman&Hall.*Adventures in Stochastic processes,*by Sidney I. Resnick, Birkhauser.*Essentials of Stochastic Processes,*by Rick Durrett, Springer.*Introduction to Probability Models,*, by Sheldon M. Ross, Academic Press*Lectures on Monte-Carlo Methods,*by Neal N. Madras, American Mathematical Society*Simulation and the Monte Carlo Method*by Reuven Y. Rubinstein and Dirk P. Kroese , Wiley*Markov chain and mixing times,*by David A. Levin, Yuval Peres, and Elizabeth L. Wilmer, American Mathematical Society*Introduction to Stochastic Processes,*by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.*Stochastic Processes,*by Sheldon M. Ross, Wiley.*A first course in Stochastic Processes,*by Samuel Karlin and Howard M. Taylor, Academic Press.

**Syllabus: **This course is an
introduction to stochastic processes and Monte-Carlo methods.
Prerequisite are a good working knowledge of calculus and elementary
probability as in Stat 607, or Stat 605. And we will use from time to
time some concepts from analysis and linear algebra. One of the main goal in the class is to develop a
"probabilist intuition and way of thinking".
We will present some proofs and we will skip some others in order to provide
a reasonably broad range of topics, concepts and techniques.
We emphasize examples both in discrete and continuous time from a wide range of
disciplines, for example branching processes, queueing systems, population models,
chemical reaction networks and so on. We will also discuss the numerical implementation of Markov chains
and discuss the basics of Monte-Carlo algorithms. Among the topics
treated in the class are

Brief review of probability concepts. The limit theorems for sums of independent random variables.

Simulation of random variables. A first look at Monte-Carlo algorithms.

Markov chains on discrete state spaces (both finite and countable). Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, applications and examples.

Continuous-Time Markov chains. Definition and basic properties. Poisson Process, Birth and Death Process, Queueing models. Renewal processes.

Reversible Markov processes and Monte-Carlo Markov Chains

Martingales.

Brownian motion and applications. Elementary stochastic analysis.

**Grade :**

Regular weekly homework will be assigned.

**Homework :**

**Homework #1**(due on Friday February 2): Homework #1**Homework #2**(due on Friday February 9 ): Homework #2

**Homework #3**(due on Friday February 16): Homework #3

**Homework #4**(due on Friday February 23): Homework #4

**Homework #5**(due on Friday March 2 ): Homework #5

**Homework #6**(due on Friday March 9): Homework #6

**Homework #7**(due on Friday March 23): Homework #7

**Homework #8**(due on Friday March 30): Homework #8

**Homework #9**(due on Friday April 13 ): Homework #9

**Homework #10**(due on Friday April 20 ): Homework #10

**Homework #11**(due on ):

**Homework #12**(due on ):